-- Logs begin at Thu 2022-09-01 04:36:23 UTC, end at Thu 2022-09-01 05:55:08 UTC. -- Sep 01 05:09:31.313228 dv-cinder-836027-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 05:09:31.321667 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 05:09:31 2022] *** Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: nodename: dv-cinder-836027-8-iscsi Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: machine: x86_64 Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: clock source: unix Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: pcre jit disabled Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: detected number of CPU cores: 4 Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: current working directory: / Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: detected binary path: /usr/bin/uwsgi-core Sep 01 05:09:31.323767 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: your processes number limit is 47680 Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: your memory page size is 4096 bytes Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: detected max file descriptor number: 2048 Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: lock engine: pthread robust mutexes Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: thunder lock: enabled Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 05:09:31.324389 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 05:09:31.363011 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: Python main interpreter initialized at 0x5600945c29c0 Sep 01 05:09:31.363011 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: python threads support enabled Sep 01 05:09:31.363011 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: your server socket listen backlog is limited to 100 connections Sep 01 05:09:31.363011 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: your mercy for graceful operations on workers is 90 seconds Sep 01 05:09:31.363371 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 05:09:31.363449 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: *** Operational MODE: preforking *** Sep 01 05:09:31.363703 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: *** uWSGI is running in multiple interpreter mode *** Sep 01 05:09:31.363703 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: spawned uWSGI master process (pid: 81557) Sep 01 05:09:31.363895 dv-cinder-836027-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 05:09:31.364932 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: spawned uWSGI worker 1 (pid: 81568, cores: 1) Sep 01 05:09:31.364932 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: spawned uWSGI worker 2 (pid: 81569, cores: 1) Sep 01 05:09:31.364932 dv-cinder-836027-8-iscsi devstack@keystone.service[81557]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 05:09:35.719149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:09:35.719149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: from cryptography.utils import int_from_bytes Sep 01 05:09:35.719896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:09:35.719896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: from cryptography.utils import int_from_bytes Sep 01 05:09:35.720221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:09:35.720221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: from cryptography.utils import int_from_bytes Sep 01 05:09:35.720678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 05:09:35.720678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: from cryptography.utils import int_from_bytes Sep 01 05:09:35.770555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 05:09:35.770750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 05:09:35.770877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] command line args: [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 05:09:35.771146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 05:09:35.771234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 05:09:35.771322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ================================================================================ {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 05:09:35.771398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 05:09:35.771564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] admin_token = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.771650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] command line args: [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 05:09:35.771808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] config_dir = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.771895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 05:09:35.771979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ================================================================================ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 05:09:35.772163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] config_source = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] admin_token = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] debug = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_dir = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] config_source = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.772950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] debug = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.773945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_config_append = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_dir = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_config_append = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_file = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.774973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_dir = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.775956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.776961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777044 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.777951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.778122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.778358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.778465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] publish_errors = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] publish_errors = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.779903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] standard_threads = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.780868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] standard_threads = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] use_journal = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] use_json = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] use_stderr = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] use_syslog = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.781944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_journal = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_json = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_stderr = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.782450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] use_syslog = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.782680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 05:09:35.782854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.782957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.783907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.784895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.785982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.786969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.787951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.788958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.789902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.790874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.791952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.792979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.793954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.794949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.795970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.796898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.797973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.798962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.799997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.800959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.801909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.802883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.803848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.804930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.805952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.806896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.external = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.807927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.external = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.password = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.808941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] auth.token = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.password = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] auth.token = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.809914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.810981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.811960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.812914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.813823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.814910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.815849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.816973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.817930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.818913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.819963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.820905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.821846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.822975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.823994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.824938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.825086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.825168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.825328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.825974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.826887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.827965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.828998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.829990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.830901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.831985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.832996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.833913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.834896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.835982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.836922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.837935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.838097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.838199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.838335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.841982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.842943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.843923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] role.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] role.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.844939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.845953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.846986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.847986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.848929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.849974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.850936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.851879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.852931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.853887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.854914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.connection = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.855888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.856954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.857947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.858938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81568) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 05:09:35.859660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.859880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 05:09:35.860055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 05:09:35.889465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.889622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.889788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.889877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.890026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.890258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.898524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.899110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.934073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.934206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.934293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:35.934392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.003272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.003440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.052275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.052508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.052733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.052974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.211346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.211519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.211705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.211925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.212141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.212351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.213461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.213692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.213918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.214142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.214362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.214606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.236559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.236668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.236795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.236884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.237880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.238005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.248970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.249163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.266960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.267581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81568) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 05:09:36.284852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.284963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.285046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:09:36.285123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:09:36.285475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.285636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.285722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:09:36.285844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 05:09:36.285930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.286986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.296954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.297969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.304746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.304952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.307680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.307864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.307976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.308851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.309949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.310790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.333657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.333952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.335155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.335445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.335578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.335721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.340129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.340342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.340473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.340630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.340773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.341188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.347914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.348207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.348337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.348507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.352330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.352471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.352622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.352706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.352912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.366744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.371263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.371504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.371589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.371783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.375579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.375754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.378729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.378929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.379025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.379223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.383240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.383625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.386527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.386713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.390151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.390463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.390555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.390759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.390849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.391052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.394724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.394845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.394935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.395026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.395117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.395117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.408290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.408584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.408671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.408859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.408947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.409131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.409218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.409406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.416048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.416164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.417587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.417800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.419210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.419387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.423599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.423674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.423916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.424051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.424128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.424218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.434664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.434840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.434976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.435832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.436000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.436111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.436295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.436391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.436582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.452329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.452508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.455274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.455526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.458183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.458477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.459781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.459914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.461267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.461406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.461563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.461660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.461852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.461939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.469189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.469460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.470786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.471106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.473640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.474841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.482596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.482804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.482892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.483100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.483183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.483387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.490118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81568) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.490220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 05:09:36.490399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.490507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.490712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.490812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.490991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.491899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.492755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81568) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.493840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 05:09:36.529159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600945c29c0 pid: 81568 (default app) Sep 01 05:09:36.529280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5600945c29c0 pid: 81569 (default app) Sep 01 05:09:36.532846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff91ee-8ff9-434f-962b-eba8d8fc940e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:36.533246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff91ee-8ff9-434f-962b-eba8d8fc940e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:36.533508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff91ee-8ff9-434f-962b-eba8d8fc940e None None] PATH_INFO: `/v3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:36.537691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Sep 1 05:09:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:44.130886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:44.131409 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:44.131517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:44.135881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:44.141546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:44.141995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:44.142090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:44.213422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: INFO dbcounter [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] Registered counter for database keystone Sep 01 05:09:44.234139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG oslo_db.sqlalchemy.engines [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81568) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 05:09:44.240331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writer thread running {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 05:09:44.418472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG passlib.handlers.bcrypt [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81568) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 05:09:44.419433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG passlib.handlers.bcrypt [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81568) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 05:09:44.477832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:44.479465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:44.606765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 05:09:45.140084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:45.209478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: INFO dbcounter [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] Registered counter for database keystone Sep 01 05:09:45.228167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG oslo_db.sqlalchemy.engines [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81569) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 05:09:45.234313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writer thread running {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 05:09:45.352676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa9d38-f2f1-4654-bd15-10c65817c773 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:45.357442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:45.358063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:45.358340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:45.358642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:45.366578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:45.417406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 05:09:45.417692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2e559-c96a-4db0-8633-15b0df2106ca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:45.427416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 1 05:09:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:09:45.433406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:45.448925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2c78b-5d0b-4947-beba-41ef2241f988 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:45.453364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:45.454480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:45.454768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:45.455045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] PATH_INFO: `/v3/projects/fd94b8179b9d4ba6806f99509678d7fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:45.456620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] RBAC: Authorizing `identity:get_project(project_id=fd94b8179b9d4ba6806f99509678d7fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:45.509005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 05:09:45.509353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3623337-b28e-4721-aeaf-9a589d645852 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:45.513896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Sep 1 05:09:45 2022] GET /identity/v3/projects/fd94b8179b9d4ba6806f99509678d7fb => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:47.364393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7926-b15a-4720-bc56-b23c356dbdda None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:47.364939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7926-b15a-4720-bc56-b23c356dbdda None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:47.364939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d7926-b15a-4720-bc56-b23c356dbdda None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:47.369000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:47.374272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:47.374717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:47.374871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:47.395146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:47.396685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:47.405086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 05:09:47.672349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d7926-b15a-4720-bc56-b23c356dbdda None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:47.684478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d7926-b15a-4720-bc56-b23c356dbdda None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:47.688715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:47.689312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:47.689595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:47.689871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:47.697452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:47.699273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36c0a5-d8e7-4e16-acbe-5c7adef54cab admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:47.708497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 1 05:09:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:09:47.713773 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:47.723494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cefde0-472c-49bc-abce-489c2d76ce0d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:47.727845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:47.728466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:47.728747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:47.729021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:47.730361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:47.732287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2855c5-013b-45fe-be86-f7275dde3b1b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:47.733803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Sep 1 05:09:47 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:49.675750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaea1c-8f95-46db-b746-2f6a2b3d59d9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:49.676267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaea1c-8f95-46db-b746-2f6a2b3d59d9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:49.676357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaea1c-8f95-46db-b746-2f6a2b3d59d9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:49.677654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:49.682907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:49.683323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:49.683572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:49.702339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:49.703540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:49.734502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ff41-64c0-4c31-aae4-c4dcf46ab6aa None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:49.734791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ff41-64c0-4c31-aae4-c4dcf46ab6aa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:49.735070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122ff41-64c0-4c31-aae4-c4dcf46ab6aa None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:49.736390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:49.741258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:49.741544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:49.741824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:49.782242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:49.783986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG passlib.handlers.bcrypt [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81569) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 05:09:49.784384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG passlib.handlers.bcrypt [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81569) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 05:09:49.833811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:49.835455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:49.936008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:09:50.045917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.069634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.148453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db66d9a-cc34-4cac-ba75-2538d22080ea None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.181568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.182199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.182509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.182790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.196014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:09:50.202772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:09:50.203780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:09:50.204717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:09:50.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.209493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.290200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.308272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c105bf-4372-4c63-8852-8a2edfe370cc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.308434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d482-59fd-4d1d-bc17-8eaa73faae84 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.311359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.342332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.343024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.343308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.343584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.344261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:50.344570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.345630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.345974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.346516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.346801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.347076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.358279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:50.363778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.413448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.415129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.459307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eaca5d-3813-4fc9-91c3-d476e7e8c5ce None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.493026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.493641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.493921 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.494190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.495359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.496837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.498387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 9/16] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:50.503895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.511630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 8/17] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:09:50.517143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.518078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.520602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.602015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791589e0-04f8-432b-b001-e148b9541e79 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.614682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3e86b-611a-4ae1-9c80-8f886c1dd8e3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.634987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.635616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.635896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.636168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.638679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.640459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.642167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:50.647964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.648866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.649724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.650006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.650276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.669170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.670675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:09:50.688755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:09:50.690094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:09:50.690960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:09:50.690960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:09:50.690960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:09:50.690960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:09:50.695560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:50.737471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.751307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:50.756348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7de34-f814-4a74-8f4d-02f5f4620304 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.791262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.791864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.792143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.792414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.793132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.domain.id=default, user.id=ad7989abb40e4e979f3e309692f0a847) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:50.793462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.794970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004de4ca-4e9d-4fd8-9a85-99869ff7247b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.795586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8184525-1ea6-4e21-9171-17c4021fa7a9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:50.801452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/role_assignments?user.id=ad7989abb40e4e979f3e309692f0a847&scope.domain.id=default&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:50.830983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:50.831612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:50.831889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:50.832159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:50.832820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:50.833129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:50.834615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e62168-c061-44f5-a500-175c20f7b9fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:50.854601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:52.848494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978a0f-ee3c-45ae-baff-6333e2e134c0 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:52.849055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978a0f-ee3c-45ae-baff-6333e2e134c0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:52.849055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa978a0f-ee3c-45ae-baff-6333e2e134c0 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:52.850734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:52.856303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7e5c-43b0-41fd-8d22-c84d6addca23 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:52.856741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7e5c-43b0-41fd-8d22-c84d6addca23 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:52.856886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7e5c-43b0-41fd-8d22-c84d6addca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:52.883352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-108b7e5c-43b0-41fd-8d22-c84d6addca23 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:52.884605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-108b7e5c-43b0-41fd-8d22-c84d6addca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:52.964689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:53.019052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041cc7-eb9b-4138-868d-3db66c3ed537 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.019334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041cc7-eb9b-4138-868d-3db66c3ed537 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.019604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff041cc7-eb9b-4138-868d-3db66c3ed537 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.021228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:53.026574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180991f-c3d5-4cd8-b3cc-8c4606c79e16 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.026865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180991f-c3d5-4cd8-b3cc-8c4606c79e16 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.027141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180991f-c3d5-4cd8-b3cc-8c4606c79e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.055427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c180991f-c3d5-4cd8-b3cc-8c4606c79e16 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:53.056629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c180991f-c3d5-4cd8-b3cc-8c4606c79e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:53.112531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:53.148276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 12/25] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:09:53.404694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041cc7-eb9b-4138-868d-3db66c3ed537 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:53.531212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff041cc7-eb9b-4138-868d-3db66c3ed537 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:53.551488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3cd8-bcc6-48a0-a184-50c151b16bb9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.551821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3cd8-bcc6-48a0-a184-50c151b16bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.552046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff3cd8-bcc6-48a0-a184-50c151b16bb9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.553389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:53.558196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7061-8f00-4981-9bba-e15bcb3454b7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.558529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7061-8f00-4981-9bba-e15bcb3454b7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.558810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b7061-8f00-4981-9bba-e15bcb3454b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.586380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:53.587093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.587378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.587653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.595614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-db8b7061-8f00-4981-9bba-e15bcb3454b7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:53.596838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-db8b7061-8f00-4981-9bba-e15bcb3454b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:53.608842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:09:53.614239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:09:53.615121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:09:53.616325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 14/27] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:09:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:09:53.625647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:53.637991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:53.687185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 14/28] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:09:53.688592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c52d3-9d0c-4bab-9c8e-6d22e2c3cb4c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.689075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c52d3-9d0c-4bab-9c8e-6d22e2c3cb4c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.689492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c52d3-9d0c-4bab-9c8e-6d22e2c3cb4c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.691222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 15/29] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:53.695908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c52d3-9d0c-4bab-9c8e-6d22e2c3cb4c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:53.825536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45315aea-dafa-407f-aff0-8a8fb55f1e33 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:53.828473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c52d3-9d0c-4bab-9c8e-6d22e2c3cb4c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:53.836551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:53.866088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:53.867565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.867983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.868446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.881038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:53.882734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:53.885664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:53.886286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.890537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.890841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.891544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:53.891904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:53.893630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a27b9a-f171-4017-b440-3192efdde631 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:09:53.900368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:09:53.901258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:09:53.902070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:09:53.902070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:09:53.902070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:09:53.902537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:09:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:53.904664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 16/31] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:09:53.907748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:53.908234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:53.908696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:53.911604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:53.945285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:53.950341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:54.067281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:54.078775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 16/32] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:54.083409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.094115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedf6d14-9a7e-4547-9a2b-dddcf87bc204 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.162798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.164126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.164773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.166641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.167510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:54.169425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.171070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.193499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 17/33] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:54.194825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.215803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:54.303639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f1a93-5aa6-4994-9afb-8d0a3ae35fe8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.320359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e530f4-9553-4aac-b17a-2c1fe6e6466e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.342538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.343529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.344008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.344477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.345480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.347128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.387789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.388743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.391169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.391169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.391169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.392876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.395498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 17/34] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:09:54 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:54.396573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.468286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:54.509275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda5b88-690a-497c-888b-7973f3f06403 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.545739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.546357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.546668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.546991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.547651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.549290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828dcde7-1dc1-420f-a6e8-cfa08659194e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.598832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 18/35] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 05:09:54 2022] POST /identity/v3/roles => generated 219 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:09:54.600094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.704640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:54.728482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf5-ebc3-4162-93bd-4ada76685636 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.764870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.765574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.765856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.766126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.767735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.769580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.771198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 19/36] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:54.772000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.791489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:09:54 2022] POST /identity/v3/projects => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:09:54.792978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0a8dd-4837-4b8e-baf2-091683084e70 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.793283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0a8dd-4837-4b8e-baf2-091683084e70 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.793567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0a8dd-4837-4b8e-baf2-091683084e70 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.795013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 19/38] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:54.795970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ead9c-f7b9-4708-b74f-846d7e07d21a None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.796230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ead9c-f7b9-4708-b74f-846d7e07d21a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.796503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ead9c-f7b9-4708-b74f-846d7e07d21a None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.797954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 20/39] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:54.798506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ead9c-f7b9-4708-b74f-846d7e07d21a None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:54.894793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f1492-f8d0-452c-8da0-1a4189b146c2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.920476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ead9c-f7b9-4708-b74f-846d7e07d21a None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:54.935389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.936336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.936808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.937187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.937880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.939927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd79778-a6de-4296-b2a6-2d138ca81079 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:54.941595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:54.960607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:54.961335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:54.961702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:54.961996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] PATH_INFO: `/v3/domains/default/users/ad7989abb40e4e979f3e309692f0a847/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:54.966035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ad7989abb40e4e979f3e309692f0a847, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:54.967719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c7889-ec1b-4d80-976a-1251fb49a613 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:55.048846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 20/40] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 05:09:54 2022] POST /identity/v3/roles => generated 225 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:09:55.050171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:55.050489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:55.050770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:55.065382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:55.078466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 21/41] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Sep 1 05:09:54 2022] PUT /identity/v3/domains/default/users/ad7989abb40e4e979f3e309692f0a847/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:09:55.078466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:55.078466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:55.078864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:55.078951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:55.079029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:55.102651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:55.104722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:55.158426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 21/42] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:55.199741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 22/43] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:55.436415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:55.469203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:55.538067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a938-221e-4a24-b6f7-40c911154d48 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:55.571084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86924e-daae-4d8b-a5b9-996e1385bf61 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:55.573637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:55.574495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:55.574780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:55.575053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:55.576161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:55.577620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:55.578591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:55.579519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 22/44] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:55.584846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:55.588574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:55.607580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:55.608204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:55.608489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:55.608764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:55.609452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:55.610889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ac522-4177-4904-bc5b-76a32c18ee25 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:55.682654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 05:09:55 2022] POST /identity/v3/roles => generated 223 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:09:55.696515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ce29f-657b-4b5a-98a4-9670aba2ba82 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:55.735015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:55.735015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:55.735015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:55.735015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:55.735430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:55.735430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bcabe-6193-4630-bfee-03f3ace690b3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:55.828331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:09:55 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:09:57.025248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23636829-4a8a-4080-88fb-c7c01ff48b36 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.025767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23636829-4a8a-4080-88fb-c7c01ff48b36 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.025850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23636829-4a8a-4080-88fb-c7c01ff48b36 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.027171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:57.032830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.033450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.033450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.051666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:57.052828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:57.101426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:57.117861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:09:57.382816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23636829-4a8a-4080-88fb-c7c01ff48b36 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:57.591641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23636829-4a8a-4080-88fb-c7c01ff48b36 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:57.624481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:57.625109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.625396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.625674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.635870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-340a7a55-a7dc-4114-8402-214d945b60d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:09:57.641308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:09:57.642172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:09:57.642981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:09:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:09:57.648604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:57.717838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bf47-e1e8-450d-a916-82beaaa9d847 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.718129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bf47-e1e8-450d-a916-82beaaa9d847 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.718431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882bf47-e1e8-450d-a916-82beaaa9d847 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.719783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 26/50] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:09:57.724738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.725029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.725311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.742690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:09:57.743908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:09:57.747099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59462c96-19ed-4876-a0b3-2ce4e39fea77 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:57.764577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:57.780504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:57.781115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:57.781394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:57.781668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:57.782335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:57.782669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:57.784109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:57.796387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 25/51] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:09:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:57.801937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:57.811123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 27/52] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:09:57.954269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:58.004051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e082-a6a6-4955-99c5-47d04ce94f44 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:58.036301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:58.036929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:58.037213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:58.037486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:58.038879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:58.040319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:58.041888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:09:57 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:58.047995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:58.085881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:58.148750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e3ca5-70b4-403b-b5f9-10b9d605358b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:58.181544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:58.182174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:58.182476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:58.182748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:58.182987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b91cb-81c9-46fc-baf1-383d09a8c5fc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:58.183923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:58.185410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:58.187002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 28/54] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:09:58.192635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:58.200635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:58.221787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:09:58.221917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:58.221917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:58.221917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:58.221917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:58.222260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:58.222260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:58.223606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:09:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:58.229009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:09:58.291674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119a24e-ea5f-4aa8-ad67-46d6f7d4f2c0 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:58.325540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:58.325782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b21ee8-c88d-4e31-abf4-71b8215461b2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:09:58.326135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:58.326425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:58.326712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:58.327433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.domain.id=default, user.id=ad7989abb40e4e979f3e309692f0a847) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:09:58.327733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:58.329184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664954ff-7232-4f80-9d73-5778b42215d3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:58.334857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 29/56] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 1 05:09:58 2022] GET /identity/v3/role_assignments?user.id=ad7989abb40e4e979f3e309692f0a847&scope.domain.id=default&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:09:58.360388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:09:58.361034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:09:58.361316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:09:58.361592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:09:58.362265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:09:58.363812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a7f24-ca47-4cf3-8364-23f771e5984b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:09:58.483971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 28/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 1 05:09:58 2022] POST /identity/v3/projects => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:10:00.370076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efd0f-7238-4dc3-b3cd-6343da39db4f None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:00.370649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efd0f-7238-4dc3-b3cd-6343da39db4f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:00.370649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efd0f-7238-4dc3-b3cd-6343da39db4f None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:00.372006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 30/58] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:00.376960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:00.377333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:00.377513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:00.395792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:00.396980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:00.425921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:00.462034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 29/59] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:00.727183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efd0f-7238-4dc3-b3cd-6343da39db4f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:00.812650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:00.825369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efd0f-7238-4dc3-b3cd-6343da39db4f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:00.857117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:00.857753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:00.858052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:00.858577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:00.859788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:00.861594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749faef3-8cc1-4dfa-a13a-48d29369d62e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:00.863163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 31/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:00.868619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:00.965890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72960d18-b19c-4312-a43b-30055e0c26ab None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:00.999695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:01.000301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:01.000585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:01.000893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:01.001557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:01.003050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463b511-63d4-41ab-9e43-75d0a92a3eeb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:01.086541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 30/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:00 2022] POST /identity/v3/users => generated 275 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:03.986539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29841c-0de7-4fee-a9d2-7b9e9eeca6e6 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:03.987028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29841c-0de7-4fee-a9d2-7b9e9eeca6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:03.987028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29841c-0de7-4fee-a9d2-7b9e9eeca6e6 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:03.988339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 32/62] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:03.993510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9209ac-3776-48e2-aa05-21671346de57 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:03.993997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9209ac-3776-48e2-aa05-21671346de57 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:03.994074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9209ac-3776-48e2-aa05-21671346de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.017371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6a9209ac-3776-48e2-aa05-21671346de57 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:04.018846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6a9209ac-3776-48e2-aa05-21671346de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:04.045258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:04.108602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 31/63] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:04.189789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef1dc-d73c-48ef-8af4-b468102d8198 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.190096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef1dc-d73c-48ef-8af4-b468102d8198 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.190351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef1dc-d73c-48ef-8af4-b468102d8198 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.191970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 33/64] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:04.196764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70997042-0649-4acc-893e-19824093f0ba None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.197280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70997042-0649-4acc-893e-19824093f0ba None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.197747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70997042-0649-4acc-893e-19824093f0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.223342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-70997042-0649-4acc-893e-19824093f0ba None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:04.224828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-70997042-0649-4acc-893e-19824093f0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:04.312082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 32/65] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:04.385535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad6c-2f3b-4c89-af63-619ac7dbc06b None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.386134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad6c-2f3b-4c89-af63-619ac7dbc06b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.387151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70ad6c-2f3b-4c89-af63-619ac7dbc06b None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.389054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 34/66] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:04.394157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f5be-1ee8-4fee-a74e-1e632932d66b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.394472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f5be-1ee8-4fee-a74e-1e632932d66b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.394755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706f5be-1ee8-4fee-a74e-1e632932d66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.413537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9706f5be-1ee8-4fee-a74e-1e632932d66b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:04.418840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9706f5be-1ee8-4fee-a74e-1e632932d66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:04.470265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ad6c-2f3b-4c89-af63-619ac7dbc06b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:04.499970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 33/67] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:04.533023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2db42-3a3f-4291-a72c-2852d2da7812 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.533655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2db42-3a3f-4291-a72c-2852d2da7812 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.534114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2db42-3a3f-4291-a72c-2852d2da7812 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.535667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 34/68] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:04.540618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.541063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.541486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.568473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:04.571520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:04.586728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:04.670243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70ad6c-2f3b-4c89-af63-619ac7dbc06b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:04.670884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 35/69] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:04.672046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:04.733866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:04.735861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.738840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.739280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.759577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:04.772844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:04.773920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:04.774948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:04.776858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 35/70] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:04.782568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:04.819042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:04.829403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046e276-7b7a-4b81-a8d4-6eadf74e48fe None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:04.869798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:04.871908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.872434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.873047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.887781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:04.896242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:04.897136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:04.898768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:04.902840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:04.918377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf53b3-ca13-434d-9d3a-4054d82082c5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:04.953809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:04.954453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.954731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.955005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.955676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:04.955981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:04.957389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cde39c-0f5a-48a9-8572-b235a7deeb23 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:04.965322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 36/72] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:04.966494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bda93-25f5-468e-bcc7-8857ef918687 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:04.966782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bda93-25f5-468e-bcc7-8857ef918687 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:04.967056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bda93-25f5-468e-bcc7-8857ef918687 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:04.968517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 37/73] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:04.969233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bda93-25f5-468e-bcc7-8857ef918687 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.007047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674856e9-a407-4168-a6e9-b1fd17a79238 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.040875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.041500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.041783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.042060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.047145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.052837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:05.058055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:05.061940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 37/74] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:05.062758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.071293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bda93-25f5-468e-bcc7-8857ef918687 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.105984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.106621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.106903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.107177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.107838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:05.108148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.109541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.117312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 38/75] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.122630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.122721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.157931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124bcdc-965d-4f3c-9d77-67cbd042c4da None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.190211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.190876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.191159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.191429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.201595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-df1e6783-47cf-40c3-9a33-8b6009f4a620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:05.206901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:05.207828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:05.208602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 38/76] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:05.209241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.209521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.209860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.217593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13af2e-86ab-41ae-a4c7-7236c9ec18a3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.227369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:05.228565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:05.251506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.252117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.252396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.252667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.262923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.266271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.268028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.269639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 39/77] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.270422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.297449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 39/78] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:05.298610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.350363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.388722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e1aa2-dee8-4ae8-a187-3266b8e5bbb1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.405073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a195-e409-4489-85fe-867ec393fa16 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.426686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.427697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.427982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.428406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.429079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:05.429384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.430803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.438602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 40/79] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.443184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.443263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.443475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.443780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.444037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.445193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.446714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.448447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.453058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.505467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.548841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28d7b-c1ab-4c19-a1c7-37a4d8c33b9c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.562789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11108c-1bf2-4eb7-abe7-3481ada543d7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.583066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.583210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.584036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.584332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.584611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.585286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:05.585594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.587031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.595802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.596598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.597089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.597710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.597994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.599374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.668868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.670331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.696850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c2bf4-1143-4f72-8bd2-307f82698533 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.719541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.730224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.730886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.731178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.739807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.739807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.739807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.739807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.739807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.773090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 41/83] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:05.773952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.796456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.836002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d08bc-d06b-4c7f-905d-aacc18a100ff None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.869049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.869684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.869973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.870253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.872831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a5e8-3e15-4232-b203-a7d36d02fcc5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:05.906564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:05.907119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:05.907403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:05.907681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:05.908798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.910290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.911888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 42/84] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:05.912660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:05.928706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:05.939172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:05.940724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:05.998034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.009593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e16e-b7ea-458c-9b98-c72c50efb10f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.037032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 43/85] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:06.037874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.044907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.045555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.045842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.046123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.047305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.048763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.050368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 43/86] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:05 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:06.051156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.128650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.138469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f06e9-74ba-4c11-9eb3-82d09fa976bd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.149157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a078e1-5833-48b5-8091-96281583dfca None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.171544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.172118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.172398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.172672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.181722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.182354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.182656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.182930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.184541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.185985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.187612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 44/87] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:06.192617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.217954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.238674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.240233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.289732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c99c0-7ad7-4c51-b1c3-98de22f6aabb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.322774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.323578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.323685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.323936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.324583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.326025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.328697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.338525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 44/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:06.339342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.395372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 45/89] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:06 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:06.396383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.450994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2864f2b-75c4-4539-996a-637e9f5d99c5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.489744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.490399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.490712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.490984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.492715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.494914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.497747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 45/90] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:06.498356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.503175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.525543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9f7d1-df30-49c3-8628-d4c44d4492c3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.563413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.567408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.567408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.567616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.567814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.609058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1db6b-69ae-4d62-a630-92859d353ad3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.651703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.653364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.657876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.658868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.658868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.659500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.659915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:06.660208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.661969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.672054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 46/91] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:06.672974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.767938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.799696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 46/92] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:06.801411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c0afe-75a5-4218-a1df-98a088550dba None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:06.805624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.839320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:06.846768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:06.847541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:06.847922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:06.848395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:06.849545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:06.851212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:06.853010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 47/93] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:06.854631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:06.986832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a490d-16e6-46a0-abac-d2c6334d58a3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:07.026233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:07.027078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:07.027452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:07.027821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:07.028648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=ad7989abb40e4e979f3e309692f0a847) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:07.029081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:07.030653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:07.041160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 47/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/role_assignments?user.id=ad7989abb40e4e979f3e309692f0a847&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:07.041845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:07.076594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df7670-8a19-4de4-bc05-22c5169fc965 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:07.093835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:07.157873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951e1d4-da86-4ab0-b60c-2d167aa6c4e2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:07.172631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:07.173330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:07.173571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:07.173848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] PATH_INFO: `/v3/projects/fb5d96c021e7472e85beed38f7fa4106` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:07.177323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] RBAC: Authorizing `identity:get_project(project_id=fb5d96c021e7472e85beed38f7fa4106)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:07.178903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:07.187116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 48/95] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:06 2022] GET /identity/v3/projects/fb5d96c021e7472e85beed38f7fa4106 => generated 312 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:07.193532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:07.256458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:07.257082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:07.257363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:07.257640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:07.258380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:07.258716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:07.260200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd09da-a77b-4578-aab4-76dbc3f2b915 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:07.269307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 48/96] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:07 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:07.312251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:07.374795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d38d5-2b4e-43e6-9447-25e8f9dd2430 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:07.422850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:07.423852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:07.424355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:07.424820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:07.425751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=fb5d96c021e7472e85beed38f7fa4106, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:07.426256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:07.427941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfaf5d-a21c-4cd3-9bea-30a49f9606ef None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:07.438853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 49/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:07 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=fb5d96c021e7472e85beed38f7fa4106&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:08.869205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e445c8-fe9a-4625-b1a5-89daff9053ab None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:08.870231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e445c8-fe9a-4625-b1a5-89daff9053ab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:08.870804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e445c8-fe9a-4625-b1a5-89daff9053ab None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:08.872449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 49/98] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:08.877612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8def-b933-4891-8c87-7a77dce0f9f6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:08.878015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8def-b933-4891-8c87-7a77dce0f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:08.878181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa8def-b933-4891-8c87-7a77dce0f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:08.907443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-77aa8def-b933-4891-8c87-7a77dce0f9f6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:08.908561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-77aa8def-b933-4891-8c87-7a77dce0f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:08.999204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 50/99] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:09.375812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e445c8-fe9a-4625-b1a5-89daff9053ab None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:09.431608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9e822-91cc-4116-94ac-d62081d97a5a None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.431919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9e822-91cc-4116-94ac-d62081d97a5a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.432162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9e822-91cc-4116-94ac-d62081d97a5a None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.434350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 51/100] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:09.439792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.440079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.440357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.477082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:09.478288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:09.575087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:09.595294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e445c8-fe9a-4625-b1a5-89daff9053ab None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:09.600135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 52/101] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:09.612388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:09.654421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:09.655077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.655405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.655681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.672394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ece1bec3-697a-41bb-8d18-80767431cca3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:09.677253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:09.678104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:09.679496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 50/102] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:09.684270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:09.787614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f25f0-39af-4125-bd5b-aafd82143e4e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:09.803705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36c43-a7f7-4028-8e0b-55c7631d85a5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.803998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36c43-a7f7-4028-8e0b-55c7631d85a5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.804270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de36c43-a7f7-4028-8e0b-55c7631d85a5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.806556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 51/103] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:09.807626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ebc82-1132-48de-b233-ca413d5ce401 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.807977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ebc82-1132-48de-b233-ca413d5ce401 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.808293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ebc82-1132-48de-b233-ca413d5ce401 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.809807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 52/104] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:09.813736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.814027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.826319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.835725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:09.836573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.837032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.837135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:09.837596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.838263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:09.838650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:09.839130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:09.841211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762e3c8-f861-4f23-b263-0b61f88c2618 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:09.858155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 53/105] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:09.859688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb4356-2ef2-48a4-bd25-ccdfe8930052 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:09.860155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb4356-2ef2-48a4-bd25-ccdfe8930052 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:09.860604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb4356-2ef2-48a4-bd25-ccdfe8930052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:09.918200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 53/106] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:09.919031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:09.978809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a4eb4356-2ef2-48a4-bd25-ccdfe8930052 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:09.981346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a4eb4356-2ef2-48a4-bd25-ccdfe8930052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:10.054273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.082283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 54/107] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:10.084057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415612df-9375-4ced-990d-e05aa02bbf57 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.084573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415612df-9375-4ced-990d-e05aa02bbf57 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.085044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415612df-9375-4ced-990d-e05aa02bbf57 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.086539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 55/108] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:10.091401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415612df-9375-4ced-990d-e05aa02bbf57 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.114795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.116993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9a04d-36a7-48af-8668-1d1603252fdc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.182711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.183750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.184231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.184754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] PATH_INFO: `/v3/users/8b03adbf299042fab16e1329e5b78f9d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.201529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] RBAC: Authorizing `identity:get_user(user_id=8b03adbf299042fab16e1329e5b78f9d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:10.203479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fba8b5-829f-4d0d-8cd7-ca61ce341a4a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:10.205004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 54/109] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:09 2022] GET /identity/v3/users/8b03adbf299042fab16e1329e5b78f9d => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:10.206258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.206580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.206859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.225264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:10.226520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:10.241118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415612df-9375-4ced-990d-e05aa02bbf57 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.276497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.278227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.278760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.279256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.280690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:10.282436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:10.284326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 56/110] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:10.285427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.299197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 55/111] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:10.304441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.341904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.412543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e842a5-778b-4107-b83d-ab96ebf04237 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.430930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.455821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92aa58-ffbb-4b21-a425-cc628cbdc852 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.455985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.455985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.455985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.455985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.462105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:10.467301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:10.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:10.468989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 57/112] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:10.469657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.492524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.493137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.493418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.493693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.569499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:10.571056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:10.572294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327dce4-1a6c-47bf-9b20-a289301f16ba None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.574425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.608363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.608975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.609265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.609545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.610206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:10.611712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:10.666096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.680871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 56/113] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:10.681685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.705931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 58/114] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:10:10 2022] POST /identity/v3/projects => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:10.706739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.789990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c3e8b-1650-4e99-a63e-c60663b9b707 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.821349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165902a-6bdb-4585-b5ce-5494c424e92e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.830308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.830995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.831273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.831556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.843528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-967a16b5-925d-416a-a97c-b5ad350736b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:10.848550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:10.849447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 57/115] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:10.851488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d39a-1e59-4fd5-9a65-c7b242c1e23d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.852204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d39a-1e59-4fd5-9a65-c7b242c1e23d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.852204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5d39a-1e59-4fd5-9a65-c7b242c1e23d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.852856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 58/116] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:10.853397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d39a-1e59-4fd5-9a65-c7b242c1e23d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.869076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:10.869509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:10.869826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:10.870055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:10.871417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:10.872011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:10.873533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:10.884671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 59/117] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:10.884671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:10.905927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.914494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:10.969312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5d39a-1e59-4fd5-9a65-c7b242c1e23d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:10.989668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25709d8-e4ce-4690-824e-22e388bd9273 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.012102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.012971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.013261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.013543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.024796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.024956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:11.025437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.025721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.026000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.027215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.028685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.035356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 60/118] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-19741e35-14e9-4ac0-8b73-77b7d9fa75db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:11.035486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:11.036349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:11.037173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:11.037765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.037832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 59/119] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:11.037832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.037832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.037832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.052076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:11.053284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:11.124044 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 60/120] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:11.124910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.139653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.163216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.164761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b01e7b-5022-4c79-8118-f04067995b48 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.211934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.212800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.213259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.214165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.215584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:11.216036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.218275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.229573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 61/121] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.231439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.250301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffb5c3-6421-4889-9eba-d52f33cb205e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.293627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.294292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.294608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.294892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.296179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.297651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.299602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 61/122] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.300248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.367094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a5c7a-6edf-4469-8b35-ba71fab94843 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.401433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67229369-6d1b-4035-988b-71314c185492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.402603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67229369-6d1b-4035-988b-71314c185492 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.402888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67229369-6d1b-4035-988b-71314c185492 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.403164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67229369-6d1b-4035-988b-71314c185492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.403894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67229369-6d1b-4035-988b-71314c185492 None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=8b03adbf299042fab16e1329e5b78f9d) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:11.404205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67229369-6d1b-4035-988b-71314c185492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.405663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67229369-6d1b-4035-988b-71314c185492 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.411202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 62/123] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/role_assignments?user.id=8b03adbf299042fab16e1329e5b78f9d&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.411981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67229369-6d1b-4035-988b-71314c185492 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.417434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bf0c2-2db2-4505-b52b-7f61eedc5c86 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.420879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.435332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.462833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.463799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.464078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.464921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.465624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:11.465935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.467425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.477357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 62/124] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.482050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.561930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67229369-6d1b-4035-988b-71314c185492 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.595189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451c68f-6d2d-4a89-82b2-5cb2a68ada96 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.612477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.613498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.614187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.614684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.615992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.618088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.620983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 63/125] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.645232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.646149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.646573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.646953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.648252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.649866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.652508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 63/126] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.653503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.690815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.730357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.753410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde02b50-3f8c-48e3-a745-4ef98a54e01e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.766573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9c19e-57b7-4c2f-bc9e-52da447efa1a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.798138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.799001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.799379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.799769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.809390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.810338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.810974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.811215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:11.811541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.812858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.814470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:11.829957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 64/127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:11.829957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:11.830150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:11.831198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:11.832154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:11.937330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99ea2d-256e-4cf7-9ef8-37da9bc435d1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.967089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d7f42-45cc-4ec5-8f32-ec955300bef4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:11.968785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:11.975251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:11.976549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:11.977058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:11.977602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/232b01a4248c49dda4c1e2c56d05fce3/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:11.980725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, user_id=232b01a4248c49dda4c1e2c56d05fce3, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:11.982625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.004115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.004849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.005225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.005597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.006883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.008575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.010450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 65/129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:11 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:12.015259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.053715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:12.066433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 65/130] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:11 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/232b01a4248c49dda4c1e2c56d05fce3/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:12.066433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.221397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ceaa-dfbb-4223-9762-c5632d113485 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.223509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46928-d56c-43ce-9b03-6384d03a5742 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.265558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.266397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.266853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.267265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.268890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:12.269333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.271278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.283822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 66/131] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:12.285115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.289611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.290167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.290581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.290956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.292238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.294320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.296224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 66/132] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:12 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:12.296950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.343330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:12.363146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:12.408726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3607c0-e63a-4b38-9aab-dd409a6cb4c1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.416970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5a6ec-7b4d-422a-9ce8-0357109ff888 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.457425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.458247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.458594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.458718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.459072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.459757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.460167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.460565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/232b01a4248c49dda4c1e2c56d05fce3/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.462954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, user_id=232b01a4248c49dda4c1e2c56d05fce3, role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.463090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.464698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.464962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.466765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 67/133] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:12 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:12.467730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.561988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 67/134] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:12 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/232b01a4248c49dda4c1e2c56d05fce3/roles/119f085c1d714b95aa1674da1868b4a6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:12.563123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.630683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f09ec-b7e9-4f3f-9dc6-73d1e3af484f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.653606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:12.724792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.725475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.725774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.726027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/ad7989abb40e4e979f3e309692f0a847/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.734786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, user_id=ad7989abb40e4e979f3e309692f0a847, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.736277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b42de-2aaf-4bd1-923e-53ab48a79b0b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.788433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24edd1-c176-448e-8e3d-d1d49d0f49f8 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.811333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:12.826772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.827339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.827569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 68/135] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:12 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/ad7989abb40e4e979f3e309692f0a847/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:12.827669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.827960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] PATH_INFO: `/v3/projects/fb5d96c021e7472e85beed38f7fa4106` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.829247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=fb5d96c021e7472e85beed38f7fa4106)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.829632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd166d6-2a40-47d0-931e-b0996eccfe5b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.830102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd166d6-2a40-47d0-931e-b0996eccfe5b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.830578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd166d6-2a40-47d0-931e-b0996eccfe5b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.830858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:12.832321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 69/136] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:12.833102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 68/137] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:12 2022] GET /identity/v3/projects/fb5d96c021e7472e85beed38f7fa4106 => generated 312 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:12.838587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:12.840492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.840784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.841542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.865610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:12.866864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:12.955177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1feec-a5ca-4b25-810a-75e2db2cc7c2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:12.990028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 70/138] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:12.993706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:12.994695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:12.995219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:12.995776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] PATH_INFO: `/v3/projects/fb5d96c021e7472e85beed38f7fa4106/users/232b01a4248c49dda4c1e2c56d05fce3/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:12.997952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] RBAC: Authorizing `identity:create_grant(project_id=fb5d96c021e7472e85beed38f7fa4106, user_id=232b01a4248c49dda4c1e2c56d05fce3, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:12.999714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ee6e8-d4a3-48e9-8b42-783d1ef3ba4d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:13.053426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 69/139] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:12 2022] PUT /identity/v3/projects/fb5d96c021e7472e85beed38f7fa4106/users/232b01a4248c49dda4c1e2c56d05fce3/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:13.517251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:13.539030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27b31-3556-4948-9554-d8d29c7344c4 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:13.539978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27b31-3556-4948-9554-d8d29c7344c4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:13.540456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c27b31-3556-4948-9554-d8d29c7344c4 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:13.542215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 70/140] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:13.548215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:13.548753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:13.549216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:13.575207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:13.582427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:13.635540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:13.663929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 71/141] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:13.687160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:13.702742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9773c-0a5e-4199-bc2a-8cc514cacd16 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:13.749882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:13.750552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:13.750833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:13.751106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:13.752255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:13.753710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:13.755631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 71/142] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:13.761309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:13.892272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee147-390a-4bc3-8878-eda1e523b911 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:13.940850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:13.941827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:13.942395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:13.942881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:13.943731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:13.945445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:14.047670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 72/143] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:13 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:14.152497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:14.270449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56a0f0-d203-413a-82ca-41a10ca8a43d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:14.310548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:14.311584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.312061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.312551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.329379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:14.334538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:14.335391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:14.336159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:14.338730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 72/144] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:14.346032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:14.389119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:14.471042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bca2c-90a0-4ffa-aa89-243a161ed834 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:14.521733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:14.522357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.522658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.522931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.523611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:14.523914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:14.525337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:14.537882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 73/145] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:14.544012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:14.601667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:14.688751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876f9a4-7396-4283-a349-e1304dd343fd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:14.733569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:14.734146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.734443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.734719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] PATH_INFO: `/v3/users/8b03adbf299042fab16e1329e5b78f9d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.735951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=8b03adbf299042fab16e1329e5b78f9d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:14.737435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9fa43-890c-4810-bafc-49df8ac96ad3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:14.739077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 73/146] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:14 2022] GET /identity/v3/users/8b03adbf299042fab16e1329e5b78f9d => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:14.747554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:14.864515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe16b-d5f8-4270-bd4b-62e588453a3e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.864836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe16b-d5f8-4270-bd4b-62e588453a3e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.865188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe16b-d5f8-4270-bd4b-62e588453a3e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.867452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 74/147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:14.874606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4190-2253-4f4d-b56d-80ea0fe11be0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.874887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4190-2253-4f4d-b56d-80ea0fe11be0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.875158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e4190-2253-4f4d-b56d-80ea0fe11be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.882751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:14.894425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ad6a9-8a18-4134-827c-80e3d881d144 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:14.919685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a85e4190-2253-4f4d-b56d-80ea0fe11be0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:14.921911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a85e4190-2253-4f4d-b56d-80ea0fe11be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:14.969964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:14.971037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:14.972489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:14.973092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:14.975748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:14.978137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:14.982795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 74/148] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:14 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:14.989400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:15.063575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 75/149] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:15.143923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac41b6-2fc6-4018-a558-ffe44f7370a1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:15.185179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:15.185856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.186136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.186422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/8b03adbf299042fab16e1329e5b78f9d/roles/b1efbfa1a5df4863bc966208c5270f82` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.191687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, user_id=8b03adbf299042fab16e1329e5b78f9d, role_id=b1efbfa1a5df4863bc966208c5270f82)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:15.193187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94bba8-5564-48f8-a17e-da7cfb84bc5c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:15.253334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 75/150] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:14 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/users/8b03adbf299042fab16e1329e5b78f9d/roles/b1efbfa1a5df4863bc966208c5270f82 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:15.315081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b42-e835-42cb-89f8-ee62c27fea68 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.315377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b42-e835-42cb-89f8-ee62c27fea68 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.315650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26393b42-e835-42cb-89f8-ee62c27fea68 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.317222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 76/151] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:15.322451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.322755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.323029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.351493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:15.352738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:15.373647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:15.428267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 76/152] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:15.576781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393b42-e835-42cb-89f8-ee62c27fea68 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:15.604582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:15.709487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26393b42-e835-42cb-89f8-ee62c27fea68 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:15.750084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:15.750763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.751036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.751303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.762805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d83f5cc6-ed8c-4531-b6ae-b36b92ad1b1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:15.768027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:15.768916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:15.769730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 77/153] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:15.778523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:15.897929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81562-d02e-4589-8e38-4ef9e4defc5d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.898244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81562-d02e-4589-8e38-4ef9e4defc5d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.898454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a81562-d02e-4589-8e38-4ef9e4defc5d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.899994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 78/154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:15.908846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.909146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.909381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.926399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2527e0-edc2-40c3-bf17-302b8e5d45b5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:15.940082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:15.941738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:15.989148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:15.989777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:15.990058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:15.990334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:15.991045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:15.991351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:15.992765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:16.009930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 77/155] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:16.010831 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.028683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 79/156] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:16.029910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.036399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.048882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.149247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a576dc-9a09-4a98-aedd-b00695beb3b4 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:16.183361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5941bd-89d9-45d8-b9af-8a49c5ad63f5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:16.207134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:16.207787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.208071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.208437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.223768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7201616d-324f-4126-b119-da9c48f6e87d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:16.229130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:16.230123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:16.231025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 78/157] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:16.235592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93572a90-4a9e-493d-ab05-ad48172da408 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.235876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93572a90-4a9e-493d-ab05-ad48172da408 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.236151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93572a90-4a9e-493d-ab05-ad48172da408 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.237545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 79/158] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:16.242771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.243058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.243333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.253173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:16.254097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.254558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.255000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.257549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:16.259255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:16.261746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 80/159] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:16.262760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.272550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:16.273760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:16.375808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.415945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 80/160] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:16.416729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.425658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.429960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d0c8e-154f-4f07-8096-7cadadca7e3b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:16.502953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:16.503795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.504310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.504652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.505366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:16.505773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:16.507729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:16.528817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 81/161] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:16.534728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.560992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a8a72-f965-4545-9d5b-8f9019323cca None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:16.614121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:16.615014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.615800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.616215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.743477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f87c18-b3f3-4295-9cd0-7a124261c750 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:16.751527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:16.753159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:16.763286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.823127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:16.823824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:16.824100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:16.824368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:16.825610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:16.827108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:16.830164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 82/162] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:16.830164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.855327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:16.945345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 81/163] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:16.946857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:16.984389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cd945-3ef6-4022-bad5-103a82120b84 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.048385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.049354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.049960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.050801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.076908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:17.084508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:17.085396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:17.086313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:17.097849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 83/164] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:17.099707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.164631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e149-37c2-4f2b-8332-16c09378ec94 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.178732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:17.232229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.233375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.233935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.234459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.238867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:17.303455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ef7f4-f2e5-4388-9e85-9b07329b8ad7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.339471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.340132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.340409 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.340680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.341819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:17.343312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:17.345420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 84/165] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:17.346300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.379581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:17.381498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:17.514117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387eede-f7e6-4b03-a003-bf7c7d0e0625 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.560361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 82/166] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:17.561228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.569628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:17.578360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.579029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.579302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.579574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.586884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:17.597014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:17.605309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:17.606260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:17.607173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 85/167] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:17.607861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.668360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63527a-f400-4f72-bc2b-c97ec1dd4b51 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.719597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.720219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.720495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.720762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.721441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:17.721740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:17.723938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:17.748213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 83/168] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:17.753092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.804730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1069d-584c-4127-8044-aaf7945229fe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:17.878145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:17.881206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:17.881500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:17.881825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:17.882592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:17.882903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:17.884378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:17.887691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:17.904810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 86/169] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:17.905574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:17.969432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e576-5f53-4c2d-9250-e31bb4815dc1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.030267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:18.063331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.063988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.064264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.064697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.069453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.071433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.074431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 84/170] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:18.075065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.101238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e355a-4ae5-4da5-8cd3-01f3fa357e7c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.174978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.175512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.175787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.176052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.176722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:18.177020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.178458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.198050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 87/171] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:18.199492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.316600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb6eb1-28f8-4183-8f19-c471c40006e2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.322432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38827-3474-408e-aec9-ffd0a7ba5c3a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.344184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:18.361297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.361923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.362202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.362551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.363324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:18.363634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.370864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.380094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.381078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.381612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.382088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] PATH_INFO: `/v3/users/232b01a4248c49dda4c1e2c56d05fce3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.383551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] RBAC: Authorizing `identity:get_user(user_id=232b01a4248c49dda4c1e2c56d05fce3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.385263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.387171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 85/172] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/users/232b01a4248c49dda4c1e2c56d05fce3 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:18.388220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.390667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 88/173] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:18.394822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.395996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:18.539985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb3a49-50f6-4e59-9ca5-5eac6f8c2f81 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.546359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20b718-9781-438d-9884-894e4f2699c0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.587142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.587805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.588086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.588359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] PATH_INFO: `/v3/users/ad7989abb40e4e979f3e309692f0a847` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.589561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] RBAC: Authorizing `identity:get_user(user_id=ad7989abb40e4e979f3e309692f0a847)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.591198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20571-c2f6-452f-90e3-6d39c62a9a05 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.592490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.593113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.593221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 86/174] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/users/ad7989abb40e4e979f3e309692f0a847 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:18.593387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.593654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.594660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23226524-1186-4499-817c-39de0efe3fab None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.594950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23226524-1186-4499-817c-39de0efe3fab None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.595287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23226524-1186-4499-817c-39de0efe3fab None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.596502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 87/175] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:18.602256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.602608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.602846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.629226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:18.630949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:18.677230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:18.702194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:18.704719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6192f1-acb1-48f3-bf9a-ff1b30d989c3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:18.744940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 88/176] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:18.746301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.748574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:18.870607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 89/177] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:18.871726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9569ed-56d2-48c2-9608-0eb14ff31ab1 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.872018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9569ed-56d2-48c2-9608-0eb14ff31ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.872295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9569ed-56d2-48c2-9608-0eb14ff31ab1 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:18.873554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 90/178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:18.879318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9569ed-56d2-48c2-9608-0eb14ff31ab1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:18.935953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd07e0-d70e-4a6a-876e-4a18f84b7f3d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:18.998114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:18.998906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:18.999094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:18.999341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.037605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.059901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9569ed-56d2-48c2-9608-0eb14ff31ab1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.099172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.100723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbd597-7cf8-4116-8b9d-db57ed89a82b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.100947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.101587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.101858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.102124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] PATH_INFO: `/v3/projects/fb5d96c021e7472e85beed38f7fa4106` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.102627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.103320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=fb5d96c021e7472e85beed38f7fa4106)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.106237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d458c16-b83f-44c1-af10-6e2c0b297ff9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.108548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 91/179] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/projects/fb5d96c021e7472e85beed38f7fa4106 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:19.109784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.110054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.110326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.129106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:19.130212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:19.193821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 92/180] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:19.194662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.210282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 89/181] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:19.211455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567025c7-aec1-4e81-ba98-680fcbf16991 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.211746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567025c7-aec1-4e81-ba98-680fcbf16991 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.212025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567025c7-aec1-4e81-ba98-680fcbf16991 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.213272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 90/182] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:19.214322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a673-60a5-4649-bd68-034b70b26fbb None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.214630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a673-60a5-4649-bd68-034b70b26fbb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.214905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512a673-60a5-4649-bd68-034b70b26fbb None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.216235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 91/183] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:19.219492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a673-60a5-4649-bd68-034b70b26fbb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.284109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.300920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01fe00-5b00-4b7c-b451-166806d40648 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.327061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512a673-60a5-4649-bd68-034b70b26fbb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.328857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.337036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.337723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.337991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.338257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.348868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:19.353601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:19.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:19.355934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 93/184] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:19.356615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.367722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.368321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.368597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.368865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.369589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=fb5d96c021e7472e85beed38f7fa4106, user.id=232b01a4248c49dda4c1e2c56d05fce3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:19.369893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.371941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b4148-e05c-4aca-b2e2-da9e6fbead13 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.377707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 92/185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/role_assignments?user.id=232b01a4248c49dda4c1e2c56d05fce3&scope.project.id=fb5d96c021e7472e85beed38f7fa4106&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:19.378697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbe25a-6528-44c9-aad9-5bbb5fabea5f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.378982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbe25a-6528-44c9-aad9-5bbb5fabea5f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.379289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbe25a-6528-44c9-aad9-5bbb5fabea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.403983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e7bbe25a-6528-44c9-aad9-5bbb5fabea5f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:19.405632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e7bbe25a-6528-44c9-aad9-5bbb5fabea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:19.487230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 93/186] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:19.496683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.496966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.497238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.497802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19a8a5-23c0-47fa-b1d6-aae611d5c012 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.517814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:19.519052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:19.550513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.551281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.551562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.551996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.553138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.555209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.556841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 94/187] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:19.559451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.578817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.615052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 94/188] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:19.616366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.661262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.702988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daf5a5-95be-4c86-b68a-36f56628397d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.738692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278729d-0631-4fc0-9b14-d21b776c4853 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.749311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.750630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.751209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.751659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.752665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:19.753146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.755372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.764680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 95/189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:19.769429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.774380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.775088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.775372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.775648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.786671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:19.791533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:19.793015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 95/190] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:19.795145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.857791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.880432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c478472-a466-4113-93a7-c5dd54c462fa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.887485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:19.904694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586569d-391a-4729-a46d-16571a9b5648 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:19.915870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.916477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.916758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.917062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.917796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=ad7989abb40e4e979f3e309692f0a847) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:19.918105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:19.919581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:19.925351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 96/191] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/role_assignments?user.id=ad7989abb40e4e979f3e309692f0a847&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:19.926129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:19.938841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:19.939502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:19.939785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:19.940060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:19.952521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:19.957552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:19.958599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 96/192] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:19.959640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.055591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8af8e4-2cb9-4c2d-8788-dd54230c47d9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.093150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.093749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.094028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.094297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.095048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9d426-e631-4e14-b970-30009305d0ed None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.106271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.109574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.111099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.112779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 97/193] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.113564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.131808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.132706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.132987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.133253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.133917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:20.134217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.136189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.147316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 97/194] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.149307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.165445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.219431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82257bdb-4ad8-4375-914d-5a6e7f114d90 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.251901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b419643-eea1-407d-864c-8b882af5885a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.252726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.253360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.253634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.253933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.255053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.256502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.258053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 98/195] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.261891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.288290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.288899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.289177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.289451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.290119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:20.290459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.291894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.299769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 98/196] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.301252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.324175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.357514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c04bce-6781-4173-b531-2b1ade54c4be None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.369373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.391945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.392813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.393086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.393352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.398398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02356efc-608c-4c65-b221-f5e9b92c8298 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.431508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.432121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.432397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.434505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.435695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.437149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.438923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 99/197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.439490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.465345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.466862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.528875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.538914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916ffdb-4e6e-4964-800a-b2d6edd0b1cb None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.563388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 99/198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:20.564218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.572204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.572809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.573089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.573879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.574559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.576013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.585816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.662719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f29f1-716d-4657-af3d-ca56dad6a5c1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.694076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 100/199] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:20 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:20.694076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.700416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.701079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.701358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.701630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] PATH_INFO: `/v3/users/8b03adbf299042fab16e1329e5b78f9d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.703452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] RBAC: Authorizing `identity:get_user(user_id=8b03adbf299042fab16e1329e5b78f9d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.705009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.706932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 100/200] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/users/8b03adbf299042fab16e1329e5b78f9d => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.712443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.753010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.812303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb7418-8f38-4171-a22b-a83335509681 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.826074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7db730-1f87-4402-8299-9db883677d21 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.853374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.853995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.854275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.855916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.866276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:20.866513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:20.866684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:20.866958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:20.868086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.869700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.871935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 101/201] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:20.872786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:20.942141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:20.951991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:20.953686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:20.996153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868dcdb-4b89-49f8-a6f0-e584fd423423 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:20.997735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:21.043712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.044337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.044613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.044882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.081978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 101/202] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:21.083188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:21.123728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.125204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.195679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:21.202488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b32ba-cb0d-426b-a9ec-e8a528081380 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:21.240341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.240953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.241319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.241656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.242476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=b07eca59b9e44e9e8079135bd31ba439) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:21.242852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.244502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.250549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 102/203] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:21 2022] GET /identity/v3/role_assignments?user.id=b07eca59b9e44e9e8079135bd31ba439&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:21.251259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:21.263084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 102/204] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:21.273796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:21.274595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:21.404495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef2b64-2872-4b36-9b6e-77d0803ad127 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:21.449838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.450421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.450713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.450984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.452432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.454639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.456814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 103/205] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:21 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:21.463095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:21.477897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e368d-41d4-47b7-a0e4-da6c650d5bd5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:21.497677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:21.575470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea4651-41aa-4365-bc52-f6c6de86d891 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:21.611161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.611909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.612287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.612659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.613495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=b07eca59b9e44e9e8079135bd31ba439) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:21.613921 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.615178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.615634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.615839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.616122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.616397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.622147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 104/206] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:21 2022] GET /identity/v3/role_assignments?user.id=b07eca59b9e44e9e8079135bd31ba439&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:21.623022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.624488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d953723-f5bc-482d-8757-467d7603db77 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.626478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 103/207] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:21 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:21.639605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:21.759972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:21.771659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33816430-15c1-4c78-a7a0-e1e08f1817cc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:21.808100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:21.808721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:21.809001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:21.809275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:21.810029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41, user.id=8b03adbf299042fab16e1329e5b78f9d) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:21.810335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:21.811893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d529e91-3386-44ab-be10-c9e5dc2fbd6a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:21.818787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 105/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:21 2022] GET /identity/v3/role_assignments?user.id=8b03adbf299042fab16e1329e5b78f9d&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:22.829955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc07c2-5c10-4f37-b758-f90819e12b4c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:22.830667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc07c2-5c10-4f37-b758-f90819e12b4c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:22.830667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc07c2-5c10-4f37-b758-f90819e12b4c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:22.831908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 104/209] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:22.837194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:22.837616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:22.837720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:22.861248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:22.862458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:22.931743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465164e-dcdc-4f17-879a-cb0d7edafbad None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:22.932042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465164e-dcdc-4f17-879a-cb0d7edafbad None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:22.932199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465164e-dcdc-4f17-879a-cb0d7edafbad None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:22.933670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 105/210] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:22.938439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:22.938708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:22.938981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:22.941596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 106/211] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:22.957918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:22.959583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:23.028286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.033058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 106/212] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:23.217426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.299823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.314890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.333801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefffe9-b5cb-44d8-bf8b-ac20df5cf721 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.378789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.379932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.380353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.380762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.392780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-f16de571-5d07-47f0-b6ac-a84466982f7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:23.397616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:23.398547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:23.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 107/213] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:23.400260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9194c-fc9f-4c14-aa1d-f562b013aaef None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.400507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9194c-fc9f-4c14-aa1d-f562b013aaef None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.400780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9194c-fc9f-4c14-aa1d-f562b013aaef None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.402663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 108/214] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:23.407278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9194c-fc9f-4c14-aa1d-f562b013aaef None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.426268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254ca57-5df8-491b-98b8-1f640ce389b6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.465327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.466327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.466934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.467350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.468697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:23.470560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5625680a-4e76-4ef0-88aa-205df28b8854 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:23.472304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 107/215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:23.473738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.474021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.474294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.492943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:23.494135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:23.520438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9194c-fc9f-4c14-aa1d-f562b013aaef None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.556644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.557248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.557532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.557800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.558504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:23.558806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:23.560210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:23.561669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.568495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 109/216] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:23.569297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.570589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 108/217] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:23.578124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.591179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.692643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738db1fb-8c9f-432a-8018-1fd35597588b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.707364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752da81-5ba3-4398-b5d5-2c98e46549ea None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.729448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.730105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.730360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.730706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.731370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:23.732951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5b2b0-5dcc-4259-bdaa-d93c2d4fed6d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:23.749540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.750180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.750488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.750787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] PATH_INFO: `/v3/users/e4d74c29053d447c9905725b5b273982` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.764918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d74c29053d447c9905725b5b273982)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:23.766433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:23.768508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 109/218] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/users/e4d74c29053d447c9905725b5b273982 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:23.773991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:23.814890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 110/219] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:23 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:23.830252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f285-c324-4b7e-9b03-cac95745f5e2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.830617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f285-c324-4b7e-9b03-cac95745f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.830882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f285-c324-4b7e-9b03-cac95745f5e2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.832502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 111/220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:23.838621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.839770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.839770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.849710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.867418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:23.868740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:23.901848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d31671-eb04-4fe3-b8c7-e196c19a74f9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:23.939294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:23.940121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:23.940233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:23.940450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:23.944821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:23.957430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 112/221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:23.958268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.030254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.031840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.079716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740c051-4a0e-458d-96e1-d2b2eb858f00 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.090735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.120633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.121442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.121551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.122292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.141098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:24.146529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:24.147470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:24.148274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 113/222] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:24.152768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 110/223] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:24.158195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.158336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.225429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.285281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0fdb-1611-43dd-bf31-350743ac883d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.298908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55503e-aaed-4963-8a6f-69d8d41d92bb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.330700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.332299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.332758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.333172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.334099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:24.334566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.336099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.336503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.337225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.337610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.337971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.339283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.340905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.343060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 111/224] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.343671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.346395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 114/225] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.348425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.376237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.466517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14498d-1c2b-4b96-b561-61b603d1b30f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.473656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899ccfe-c42c-42d4-80a1-6f5d1f3d8214 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.488860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.511999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.512802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.513051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.513265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.513632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.514191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.514633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.514859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=e4d74c29053d447c9905725b5b273982) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:24.515424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.515730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.518317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.525160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 115/226] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/role_assignments?user.id=e4d74c29053d447c9905725b5b273982&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.526690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.531325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:24.536496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:24.537446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:24.538261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 112/227] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:24.547565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.675586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49680d04-e2c4-4047-bed3-d65c2ff7abc2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.712651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.713372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.714234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.714607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.715811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.717343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.719164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 116/228] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.724521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.728763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.759507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c085c30b-785a-4bef-8793-4fa873a60c59 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.805479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:24.827651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.828572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.828995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.829404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.830219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:24.830683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.832252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.842688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 113/229] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.848496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:24.868070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c364-2d92-4e10-8626-27e6fc452344 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:24.902962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:24.903687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:24.910426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:24.910426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:24.910426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:24.910426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:24.910426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 117/230] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:24.914498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:25.026752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f072f-14b5-4003-b759-28800e711eaa None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:25.047115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:25.061441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:25.062192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:25.062501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:25.062772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/b07eca59b9e44e9e8079135bd31ba439/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:25.064713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, user_id=b07eca59b9e44e9e8079135bd31ba439, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:25.066234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:25.085381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4cbc5-c527-4441-b0b1-97b275094ee1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:25.121314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:25.121955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:25.122239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:25.122545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:25.123688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:25.125183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:25.127290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 114/231] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:24 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:25.132575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:25.152496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 118/232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:24 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/b07eca59b9e44e9e8079135bd31ba439/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:25.161845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:25.295815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c824ee0-933a-4d0d-81f4-de0b962276b3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:25.336038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:25.336603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:25.336877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:25.337146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:25.338263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:25.340299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a59325-0703-48c1-9ee9-1f6b78773b89 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:25.342229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 115/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:25 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:25.347976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:25.462384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbcf75-ca3b-404f-9182-343adc255e78 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:25.505443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:25.506103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:25.506379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:25.506677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/b07eca59b9e44e9e8079135bd31ba439/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:25.512728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, user_id=b07eca59b9e44e9e8079135bd31ba439, role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:25.514273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b8ea0-e9f4-42b1-865a-7aaa0dc269d7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:25.578155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 119/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:25 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/b07eca59b9e44e9e8079135bd31ba439/roles/119f085c1d714b95aa1674da1868b4a6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:26.235700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78270c-1d99-4aff-9936-fd9728a6c635 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.236771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78270c-1d99-4aff-9936-fd9728a6c635 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.237263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78270c-1d99-4aff-9936-fd9728a6c635 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.238847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 116/235] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:26.244343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999a9a-2bea-4749-ad43-66d8dd6f4725 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.245727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999a9a-2bea-4749-ad43-66d8dd6f4725 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.246199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999a9a-2bea-4749-ad43-66d8dd6f4725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.269838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b8999a9a-2bea-4749-ad43-66d8dd6f4725 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:26.271422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b8999a9a-2bea-4749-ad43-66d8dd6f4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:26.303825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b040d6-4ec3-42a5-bb87-90b10b7c062b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.304442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b040d6-4ec3-42a5-bb87-90b10b7c062b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.304940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b040d6-4ec3-42a5-bb87-90b10b7c062b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.306375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:26.306731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 117/236] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:26.313071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed29f-9af6-4d3d-926b-5096eeea163e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.313628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed29f-9af6-4d3d-926b-5096eeea163e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.314098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ed29f-9af6-4d3d-926b-5096eeea163e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.332864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ee7ed29f-9af6-4d3d-926b-5096eeea163e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:26.334376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ee7ed29f-9af6-4d3d-926b-5096eeea163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:26.372061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 120/237] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:26.444075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 118/238] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:26.714353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d1fc-9918-4345-b01c-1fd2c2ef5df9 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.714722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d1fc-9918-4345-b01c-1fd2c2ef5df9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.714944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d1fc-9918-4345-b01c-1fd2c2ef5df9 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.716557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 121/239] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:26.721641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.721930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.722206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:26.746819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:26.748011 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:26.755445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:26.769660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1d1fc-9918-4345-b01c-1fd2c2ef5df9 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:26.837927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 119/240] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:26.839294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:26.943442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1d1fc-9918-4345-b01c-1fd2c2ef5df9 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:26.953927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d48d9-4dcd-4748-8fe2-6bc66c2968c7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:26.994906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:26.995847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:26.996620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:26.997066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.008901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:27.012157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.012848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.013129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.013402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:27.014927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:27.015788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:27.016596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:27.019103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 120/241] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:27.019206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.020672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.027274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 122/242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:27.028400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.032775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.036401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.091033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.180579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe8ea5-5165-49ec-a1fb-0f44f03e7a1d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.217458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.218387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.218897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.219358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.220236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.221888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.244672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad4470-cd5c-4c28-bb7b-9cf7efb78853 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.284657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.285384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.285596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.285818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.286803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:27.287110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.288563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.298182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 123/243] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:27.299633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 121/244] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:27 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:10:27.303361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.304308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.426783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.482357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.485250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43bc3-7dd4-44d0-8fe6-699031b64b4a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.494458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38af2f0-b184-413c-8c70-aef50a270cbb None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.531155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.532543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.533065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.533519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] PATH_INFO: `/v3/users/533f6feeac7e4aa58e31990b39bd8980` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.546148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.546945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.547220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.547511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.555854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] RBAC: Authorizing `identity:get_user(user_id=533f6feeac7e4aa58e31990b39bd8980)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.556008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7abeb-88af-4f7c-97ee-9d6cc5cb8e8f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.557851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 122/245] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/users/533f6feeac7e4aa58e31990b39bd8980 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:27.558904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ddd2-271a-4d75-ab23-44131a043cb6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.559244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ddd2-271a-4d75-ab23-44131a043cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.559512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1ddd2-271a-4d75-ab23-44131a043cb6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.564431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:27.565050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 123/246] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:27.566282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.566586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.566856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:27.570345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:27.571239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:27.572071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 124/247] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:27.573269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.587689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:27.589535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:27.671457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 124/248] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:27.672486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.707768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c69b6c-efb2-408a-8c0b-1ede26aaa28b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.739401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.743612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.744235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.744519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.745766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.772604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.784243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67879d-6b18-441d-9db6-8fa7a76dfbed None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.821356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:27.821557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.822465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:27.823623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:27.824116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:27.824363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.825219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:27.825720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:27.827325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:27.838639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 125/249] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:27.848879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:27.973839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20513a8-1c4a-485f-be1c-64fbea5dca72 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:27.984707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 125/250] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:27.986746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:27.989928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.046448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.047137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.047421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.047695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] PATH_INFO: `/v3/users/e4d74c29053d447c9905725b5b273982` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.048962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d74c29053d447c9905725b5b273982)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.050566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.052582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 126/251] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/users/e4d74c29053d447c9905725b5b273982 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:28.057808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.059041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.134054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a8be2-cfd5-4b38-9791-7514883d3341 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.191315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca779bdf-6fa8-4520-9d39-51f057f1ba18 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.227992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.229503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.229928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.230343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.238662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.239307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.239594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.239923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.241135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.242672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.242813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:28.244458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 126/252] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:27 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:28.246446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b76bba48-3a4c-4db1-b294-aeb0b5724d18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:28.258524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:28.259700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:28.260804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 127/253] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:28.261503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a61772-5443-4bec-88a5-f4dda4b75df4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.261622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a61772-5443-4bec-88a5-f4dda4b75df4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.261902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a61772-5443-4bec-88a5-f4dda4b75df4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.267576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 128/254] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:28.267576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a61772-5443-4bec-88a5-f4dda4b75df4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.309991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.365344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a89bed-69fb-4064-8d3e-64512c6c0135 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.380575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a61772-5443-4bec-88a5-f4dda4b75df4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.390556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.403260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4c878-8636-4098-a847-334b8881797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.404028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c878-8636-4098-a847-334b8881797d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.404271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c878-8636-4098-a847-334b8881797d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.404545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c878-8636-4098-a847-334b8881797d None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.405694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c878-8636-4098-a847-334b8881797d None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.407256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c878-8636-4098-a847-334b8881797d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.409049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 127/255] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:28.410446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.410446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.410969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.424248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.425700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.425979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.426249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.427010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=533f6feeac7e4aa58e31990b39bd8980) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:28.427357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.428768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.431838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:28.433289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:28.434470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 129/256] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/role_assignments?user.id=533f6feeac7e4aa58e31990b39bd8980&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:28.435284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.509971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 128/257] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:28.511383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.611445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.614307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8832f-d9a7-4854-919a-be232a6c8270 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.645327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2c581-1bd2-4679-aa03-d5310c3e2b92 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.660145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.661074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.661183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.661426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.662111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:28.662436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.663842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.684607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 130/258] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:28.690369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.710915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.756891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.757576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.757898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.758141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/e4d74c29053d447c9905725b5b273982/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.760225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, user_id=e4d74c29053d447c9905725b5b273982, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.761648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.802714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec14737-8718-40db-acce-f2e851db984a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.836963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.837602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.837881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.838154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:28.839903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:28.841362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:28.842976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 131/259] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:28.848646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.858730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 129/260] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:28 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/e4d74c29053d447c9905725b5b273982/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:28.881561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:28.944945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:28.957826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac20393-82cd-4a6c-b7c4-c821d1e87a22 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:28.997682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:28.998310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:28.998614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:28.998880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.076741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.078282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.089025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7a4b8-4b6e-44d6-912f-40bff8645b31 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.111876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:29.138431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b750a-f945-4592-b814-98873de26476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.139458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b750a-f945-4592-b814-98873de26476 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.139932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b750a-f945-4592-b814-98873de26476 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.140397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b750a-f945-4592-b814-98873de26476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.153206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857b750a-f945-4592-b814-98873de26476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-857b750a-f945-4592-b814-98873de26476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:29.159726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:29.160636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:29.161436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:29.164162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 130/261] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:29.170207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b750a-f945-4592-b814-98873de26476 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.189332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 132/262] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:29.195214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.220944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:29.285219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b750a-f945-4592-b814-98873de26476 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.302278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd43c8-2029-4e22-87c0-864969a62567 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.333976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.335195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.335692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.336506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.337623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:29.338113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.338724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:29.341940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.344794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.345501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.345849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.346257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.347660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.350916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.350916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 133/263] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:29.352993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 131/264] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:29.357199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.359917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.493239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97b40-588a-4b1f-8c5a-23c8764734d2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.495625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:29.534233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.535262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.535739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.536206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] PATH_INFO: `/v3/users/b07eca59b9e44e9e8079135bd31ba439` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.537565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] RBAC: Authorizing `identity:get_user(user_id=b07eca59b9e44e9e8079135bd31ba439)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.540182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.542868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 132/265] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/users/b07eca59b9e44e9e8079135bd31ba439 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:29.549244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.606780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1529d3-633b-4f1a-882c-e43356cd8b1f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.665907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.667068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.667349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.667621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.668349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=b07eca59b9e44e9e8079135bd31ba439) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:29.668652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.670104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.675926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49a37-dea5-4073-8c46-5e53518ac01d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:29.685294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 134/266] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/role_assignments?user.id=b07eca59b9e44e9e8079135bd31ba439&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:29.713690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:29.714683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:29.715160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:29.715627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:29.758930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:29.792077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:29.793934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:29.904970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 133/267] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:29.910843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:29.981545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:30.022472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc064ed-497d-4dca-b90b-f456deb9f975 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:30.058651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:30.059652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.060161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.060624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.061994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:30.063678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552aa251-96a9-4c1d-837b-e382542aef32 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:30.065512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 135/268] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:29 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:30.071632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:30.144129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:30.177828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4725afa-a82d-491c-b5af-2ae7c1e07bc1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:30.217448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:30.218474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.218950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.219487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.220429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=b07eca59b9e44e9e8079135bd31ba439) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:30.220947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:30.222607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d33df-2699-407d-9f68-4cd17d508b42 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:30.229589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 134/269] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:30 2022] GET /identity/v3/role_assignments?user.id=b07eca59b9e44e9e8079135bd31ba439&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:30.375314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5be45-ae9f-4398-93dc-d393bf5aa1ee None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.376300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5be45-ae9f-4398-93dc-d393bf5aa1ee None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.376771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5be45-ae9f-4398-93dc-d393bf5aa1ee None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.378676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 136/270] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:30.384151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.385726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.386198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.394741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3e77-e6c1-4cb3-ab06-4738a4f4b247 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.395405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3e77-e6c1-4cb3-ab06-4738a4f4b247 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.395871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf3e77-e6c1-4cb3-ab06-4738a4f4b247 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.397388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 137/271] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:30.403475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.403774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.404035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.415496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:30.417096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:30.439816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:30.440918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:30.502220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 135/272] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:30.546081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 138/273] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:30.789265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:30.849577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:30.862638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:30.912318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43ca6c-2e1e-44ea-aea2-99e2481ce9fb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:30.918580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:30.952239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:30.952883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:30.953287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:30.953729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:30.954967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:30.956989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:30.958836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 136/274] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:30.964058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:30.972959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76614f0-a832-4618-94b7-f7f97ee1c5e7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.009552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.010752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.011325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.024120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:31.029261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:31.030288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:31.031121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 139/275] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:31.036162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:31.086783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed2b2a-07f0-4d02-90ec-c49cc57db313 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.119208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:31.127344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.128430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.132068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.132068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.132068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:31.132640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c4fd5-b4d1-483a-823c-2ce200a09b72 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:31.160356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f954c-c5e1-4c29-9091-9df584668173 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.181880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:31.195912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.196526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.196802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.197082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.197782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:31.198081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:31.199579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:31.208177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 140/276] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:31.213670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:31.242708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 137/277] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:30 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:10:31.339081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710c3fd-4c64-45a9-99f6-6915d221aa3b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.349759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e111a-94a1-4f23-9829-50ca5feefbaf None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.350084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e111a-94a1-4f23-9829-50ca5feefbaf None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.350297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e111a-94a1-4f23-9829-50ca5feefbaf None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.351839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 138/278] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:31.358130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b32-f37b-4d49-8eda-a4a827190bd2 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.358450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b32-f37b-4d49-8eda-a4a827190bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.358715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b32-f37b-4d49-8eda-a4a827190bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.384267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a44e5b32-f37b-4d49-8eda-a4a827190bd2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:31.385529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a44e5b32-f37b-4d49-8eda-a4a827190bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:31.387841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.389022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.389519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.390004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] PATH_INFO: `/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.408170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ae8fc222fa483fbf4b7a8e0256098a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:31.408497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:31.411604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 141/279] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:31 2022] GET /identity/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:31.417242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:31.465388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:31.473504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 139/280] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:31.539288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440b168-2cf8-4bfb-99d7-fdebaa726cf1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.575873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.576541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.576877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.577153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.577947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, user.id=b6ae8fc222fa483fbf4b7a8e0256098a, scope.system=all) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:31.578254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:31.579797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c645e9-6dfe-4233-8eb7-5f5a71707d35 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:31.586148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 142/281] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 05:10:31 2022] GET /identity/v3/role_assignments?user.id=b6ae8fc222fa483fbf4b7a8e0256098a&scope.system=all&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:31.723999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0c64-4270-4914-97bb-9ee26b92b322 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.724355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0c64-4270-4914-97bb-9ee26b92b322 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.724536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0c64-4270-4914-97bb-9ee26b92b322 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.726208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 140/282] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:31.732289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.732833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.733330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.754197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:31.755715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:31.757574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0c64-4270-4914-97bb-9ee26b92b322 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:31.842650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 143/283] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:31.899242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0c64-4270-4914-97bb-9ee26b92b322 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:31.919876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:31.943074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:31.944039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:31.944569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:31.945065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:31.960298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:31.966582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:31.967509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:31.968320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:31.970327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 141/284] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:31.976343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.022269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:32.100110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf9723-c1f4-4b6d-b267-f99494225d71 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.142118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.143005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.143432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.143872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.144779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:32.145226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.145534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.147100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.158086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 144/285] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:32.163687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.262776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:32.270189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614bc7f-155e-4655-8231-b64ca9565de3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.325091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.326012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.326463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.326877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] PATH_INFO: `/v3/users/533f6feeac7e4aa58e31990b39bd8980` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.331142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] RBAC: Authorizing `identity:get_user(user_id=533f6feeac7e4aa58e31990b39bd8980)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.332782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.334104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa75bb-f92a-44e6-956c-e23c39c30c57 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.334762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 145/286] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/users/533f6feeac7e4aa58e31990b39bd8980 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:32.341771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.376684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.377323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.377605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.377880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.390662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:32.396540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:32.397607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:32.398627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 142/287] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:32.409068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.480863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacd157-b947-44f4-88eb-2c6860e1d55b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.488378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:32.524621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.525726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.526207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.526857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.529291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.532347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.534356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 146/288] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:32.541149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.600645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:32.615163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a63c2-e1c8-4d32-9147-9d9e59de012f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.680801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.685724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.686004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.686281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.687008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:32.687315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.688745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.706808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 143/289] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:32.714967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:32.745440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0161e6c-306e-438f-aa60-e3b2a997b2ed None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.799125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.799800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.800079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.800350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/533f6feeac7e4aa58e31990b39bd8980/roles/b1efbfa1a5df4863bc966208c5270f82` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.802329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, user_id=533f6feeac7e4aa58e31990b39bd8980, role_id=b1efbfa1a5df4863bc966208c5270f82)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.803794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.851163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:32.854979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fb86a-6faa-4db4-99a4-bf557ba1f1a1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:32.871231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 147/290] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:10:32 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/users/533f6feeac7e4aa58e31990b39bd8980/roles/b1efbfa1a5df4863bc966208c5270f82 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:10:32.908999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:32.909672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:32.909947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:32.910223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] PATH_INFO: `/v3/users/e4d74c29053d447c9905725b5b273982` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:32.911634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] RBAC: Authorizing `identity:get_user(user_id=e4d74c29053d447c9905725b5b273982)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:32.913215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:32.915291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 144/291] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/users/e4d74c29053d447c9905725b5b273982 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:32.921785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:33.047502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:33.056953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd46d6-c9b3-4a70-8d2f-fcd8930a4ef4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:33.092718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:33.093391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.093668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.093939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.164124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:33.165609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:33.259079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:33.268569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 148/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:33.274211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:33.429152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586bdf76-c553-40e4-b4fc-86dd0210414d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:33.475119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:33.476109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.476603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.477066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.483090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:33.484820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670d5a16-5f5b-483b-9c2a-bc3f8e4685a2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:33.486728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 145/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:33 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:33.492561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:33.632207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3699c-0b90-41f4-9aaa-78357e388e36 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:33.673923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:33.674569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.674847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.675121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.675859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=e4d74c29053d447c9905725b5b273982) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:33.676172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:33.677626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca4ca7-63a8-46ab-8f4d-e526820eabae None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:33.684470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 149/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:33 2022] GET /identity/v3/role_assignments?user.id=e4d74c29053d447c9905725b5b273982&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:33.812080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3978-4f74-4e54-82cb-073b70f4f9eb None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.812409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3978-4f74-4e54-82cb-073b70f4f9eb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.812645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b3978-4f74-4e54-82cb-073b70f4f9eb None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.814309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 146/295] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:33.820050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d017-7bf2-47e2-894b-a6fe7d1ce12d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.829037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d017-7bf2-47e2-894b-a6fe7d1ce12d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.829037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d017-7bf2-47e2-894b-a6fe7d1ce12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.829230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ea79-5ea1-439a-8db6-b24daddf7c3a None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.829230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ea79-5ea1-439a-8db6-b24daddf7c3a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.829230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ea79-5ea1-439a-8db6-b24daddf7c3a None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.829230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 147/296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:33.830418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c16e-f6df-43f3-82f9-d0d6928b65f0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:33.830725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c16e-f6df-43f3-82f9-d0d6928b65f0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:33.830998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6c16e-f6df-43f3-82f9-d0d6928b65f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:33.858813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5ae6d017-7bf2-47e2-894b-a6fe7d1ce12d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:33.860026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5ae6d017-7bf2-47e2-894b-a6fe7d1ce12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:33.867318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b2b6c16e-f6df-43f3-82f9-d0d6928b65f0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:33.868499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b2b6c16e-f6df-43f3-82f9-d0d6928b65f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:33.906290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:33.934041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:33.957278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 150/297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:33.965766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 148/298] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:34.041872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cda83-a569-4695-aa2f-3880a3f4489b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.042491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cda83-a569-4695-aa2f-3880a3f4489b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.042986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856cda83-a569-4695-aa2f-3880a3f4489b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.044554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 151/299] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:34.049812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684c879-90f4-4622-8542-223264ebed2d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.050319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684c879-90f4-4622-8542-223264ebed2d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.050814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684c879-90f4-4622-8542-223264ebed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.073022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b684c879-90f4-4622-8542-223264ebed2d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:34.074538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b684c879-90f4-4622-8542-223264ebed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:34.141942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 149/300] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:34.248781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cda83-a569-4695-aa2f-3880a3f4489b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.264626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684c879-90f4-4622-8542-223264ebed2d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.372308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:34.372499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856cda83-a569-4695-aa2f-3880a3f4489b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.379408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684c879-90f4-4622-8542-223264ebed2d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.419024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.419671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.419946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.420218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.420510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.421136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.421412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.421679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.430787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:34.432668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:34.436183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:34.437069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:34.442994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:34.443627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:34.444473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:34.445299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:34.445933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 150/301] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:34.445933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.446056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 152/302] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:34.446056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.496688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:34.552173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885657-7dbb-452a-8cd6-9a473d129948 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.560194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5316-6e57-44c4-8c96-340afeabe907 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.590121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.590756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.591036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.591301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.591956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:34.592254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.593633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.596714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.597342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.597624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.597899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.599023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.600475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.606443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 153/303] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:34.606588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 151/304] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:34.606668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.607078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.642389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:34.721933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857eec-ea98-4966-9694-65f669141bdf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.723603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973b92-1a58-4836-8932-faa54786c834 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.733646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:34.757006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.757635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.757911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.758178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] PATH_INFO: `/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.759322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] RBAC: Authorizing `identity:get_user(user_id=b6ae8fc222fa483fbf4b7a8e0256098a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.760887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.761309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.761492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.761770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.762040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.762731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:34.768077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.768197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 152/305] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:34.768197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.768354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.775363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 154/306] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:34.780007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:34.868298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:34.877121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b2547-b69a-4a47-8083-17bd4d6ca25e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.886039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad53edb-3827-4835-ab85-6822afb1675e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:34.911826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.912427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.912862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.913145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] PATH_INFO: `/v3/groups` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.913770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.915255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.920046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:34.920700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:34.920977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:34.921248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] PATH_INFO: `/v3/system/users/b6ae8fc222fa483fbf4b7a8e0256098a/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:34.922763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b6ae8fc222fa483fbf4b7a8e0256098a, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:34.924067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:34.983647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 153/307] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 1 05:10:34 2022] POST /identity/v3/groups => generated 229 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:34.983647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:35.007294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 155/308] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 1 05:10:34 2022] PUT /identity/v3/system/users/b6ae8fc222fa483fbf4b7a8e0256098a/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:35.117449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cf4d7-e12f-4009-b220-52b3798c69c1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:35.161397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:35.162304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.162821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.163234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] PATH_INFO: `/v3/users/19a471a8176b409b86c3d7b5b4355373` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.178756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] RBAC: Authorizing `identity:get_user(user_id=19a471a8176b409b86c3d7b5b4355373)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:35.180447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee7ffe-dc14-4cae-9ee4-624b83afa85c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:35.182618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 154/309] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:34 2022] GET /identity/v3/users/19a471a8176b409b86c3d7b5b4355373 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:35.188480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:35.226840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:35.248454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b5dc-a253-44b6-92ad-ebd12b3ff8e9 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.248796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b5dc-a253-44b6-92ad-ebd12b3ff8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.249000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214b5dc-a253-44b6-92ad-ebd12b3ff8e9 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.251375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 155/310] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:35.255580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.255867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.256143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.279818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:35.281845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:35.294778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:35.307036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa5a52-81a2-4836-ac53-c4e0e1de8afd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:35.347781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:35.348745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.349196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.349649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.350576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, user.id=19a471a8176b409b86c3d7b5b4355373, scope.system=all) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:35.351056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:35.352668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:35.358818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 156/311] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 05:10:35 2022] GET /identity/v3/role_assignments?user.id=19a471a8176b409b86c3d7b5b4355373&scope.system=all&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:35.366439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 156/312] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:35.634690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:35.737537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:35.741547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f6d08-6486-414b-9af9-58dd956f8502 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:35.776232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:35.777342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.777718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.778088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.789976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:35.795302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:35.796187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:35.797003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 157/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:35.802134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:35.905189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838897f8-dd99-4a95-bf66-a19e8ad3a4e4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:35.940282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:35.941262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:35.941741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:35.942220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:35.943105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:35.943661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:35.945246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:35.954278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 157/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:35.962685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:36.061918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fed6ad-0afb-435c-b758-be901cd0716c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:36.096118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:36.096750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:36.097022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:36.097290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] PATH_INFO: `/v3/users/533f6feeac7e4aa58e31990b39bd8980` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:36.098430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] RBAC: Authorizing `identity:get_user(user_id=533f6feeac7e4aa58e31990b39bd8980)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:36.099871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:36.101733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 158/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:35 2022] GET /identity/v3/users/533f6feeac7e4aa58e31990b39bd8980 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:36.107939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:36.112907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:36.216878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc492b-a2d5-43a9-b86d-098039729fc4 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:36.251538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:36.252413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:36.252865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:36.253306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:36.327317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:36.328862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:36.330721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:36.430910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 158/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:36.436828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:36.465905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:36.540657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365875c5-96cd-48bb-9363-2f492891dc70 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:36.576656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:36.577288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:36.577562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:36.577826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:36.578959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:36.580399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d012c-74e0-4b30-be88-70353999cae3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:36.582298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 159/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:36 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:36.588719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:36.702916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd7678-5757-4cd5-8aaa-bf0c5171a7ac None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:36.704822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:36.737284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:36.737897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:36.738178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:36.738478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:36.739217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb, user.id=533f6feeac7e4aa58e31990b39bd8980) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:36.739525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:36.741037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1ae2e-f087-4a56-ba29-f9cc476668de None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:36.754635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 159/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:10:36 2022] GET /identity/v3/role_assignments?user.id=533f6feeac7e4aa58e31990b39bd8980&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:37.015882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963fd19-2f38-4f6f-80e8-1640a24bff58 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.016654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963fd19-2f38-4f6f-80e8-1640a24bff58 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.016865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963fd19-2f38-4f6f-80e8-1640a24bff58 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.018176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 160/319] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:37.023245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16b6e-e448-44f3-811b-e03b8a36cdef None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.023629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16b6e-e448-44f3-811b-e03b8a36cdef None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.023792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc16b6e-e448-44f3-811b-e03b8a36cdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.042772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-fdc16b6e-e448-44f3-811b-e03b8a36cdef None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:37.043967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fdc16b6e-e448-44f3-811b-e03b8a36cdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:37.109530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 160/320] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:37.189741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1593e4-ac24-4d53-8697-fc402d023e0f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.190056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1593e4-ac24-4d53-8697-fc402d023e0f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.190287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1593e4-ac24-4d53-8697-fc402d023e0f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.191773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 161/321] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:37.196605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.196896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.197202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.215013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:37.216192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:37.249400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cbc24-39c4-452d-a6c5-3f4bfe8ecfe5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.249695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cbc24-39c4-452d-a6c5-3f4bfe8ecfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.249961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cbc24-39c4-452d-a6c5-3f4bfe8ecfe5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.251443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 162/322] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:37.256206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.256491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.256765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.275953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:37.277123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:37.288024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 161/323] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:37.344940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 163/324] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:37.387409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.415087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:37.487578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07c22e-0619-4c25-8cb2-5006a48b7d86 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.520715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.521346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.521624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.521896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.532548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:37.537349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:37.538256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:37.539129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:37.542715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 162/325] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:37.544726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.552596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.571928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:37.643553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:37.649514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e3d3-3d26-4d27-a778-1476854b7b79 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.658306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec72b28-c137-46b1-a879-296ee38f99be None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.683258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.683865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.684139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.684412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.685071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:37.685373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:37.686820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:37.691647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.692268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.692542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.692808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.693926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:37.694637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 164/326] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:37.695413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:37.695519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.698459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 163/327] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:37.702475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.782360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:37.797619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a00d3d4-6f26-454a-8b18-cbbb98567842 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.799767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf2dde-ebca-4f5b-8fdc-ae3d6ad799a8 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.831313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.831932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.832282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.832478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.833033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.833656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.833937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.834373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] PATH_INFO: `/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.835796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ae8fc222fa483fbf4b7a8e0256098a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:37.837265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:37.839587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 164/328] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/users/b6ae8fc222fa483fbf4b7a8e0256098a => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:37.840265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.847488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:37.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:37.853339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:37.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 165/329] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:37.855319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:37.864641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:37.937461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cbc00-d9c4-4fa9-a345-608378e435b2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.953290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456a24-efb2-45cd-bd3d-dc6eb6156b0d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:37.970055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.970682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.970958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.971225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] PATH_INFO: `/v3/groups` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.971847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:37.973278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:37.986390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] RBAC: Adding query filter params (role.id=2b04b45f17f1428facf4bd2291e8bcb3, user.id=b6ae8fc222fa483fbf4b7a8e0256098a, scope.system=all) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:37.991442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:38.001376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 166/330] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 05:10:37 2022] GET /identity/v3/role_assignments?user.id=b6ae8fc222fa483fbf4b7a8e0256098a&scope.system=all&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 510 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:38.002236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:38.006210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:38.040861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 165/331] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 1 05:10:37 2022] POST /identity/v3/groups => generated 226 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:38.130327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060682a-d734-4aca-8407-0419d796fb5e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:38.168979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:38.172387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:38.172387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:38.172387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:38.172387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:38.172387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:38.172765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:38.183466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 167/332] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:38.189373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:38.276277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:38.294450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888a87c-8c5f-4f02-9fe9-635fbe482813 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:38.333375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:38.334017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:38.334295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:38.334592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] PATH_INFO: `/v3/users/19a471a8176b409b86c3d7b5b4355373` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:38.335747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] RBAC: Authorizing `identity:get_user(user_id=19a471a8176b409b86c3d7b5b4355373)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:38.337211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10373655-798b-4911-91ea-aba8ba63fdb9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:38.338874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 166/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:38 2022] GET /identity/v3/users/19a471a8176b409b86c3d7b5b4355373 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:38.344500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:38.413944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:38.451651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c452506-eb32-40b6-a831-65c1f8e105f0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:38.485970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:38.486659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:38.486939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:38.487217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] PATH_INFO: `/v3/system/users/19a471a8176b409b86c3d7b5b4355373/roles/b1efbfa1a5df4863bc966208c5270f82` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:38.488648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=19a471a8176b409b86c3d7b5b4355373, role_id=b1efbfa1a5df4863bc966208c5270f82)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:38.490038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c163dbf-cb85-47c1-8a7d-441c00362569 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:38.577735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 168/334] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 1 05:10:38 2022] PUT /identity/v3/system/users/19a471a8176b409b86c3d7b5b4355373/roles/b1efbfa1a5df4863bc966208c5270f82 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:40.194507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b7ce-affe-4662-b3f3-271b00b34c18 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:40.195085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b7ce-affe-4662-b3f3-271b00b34c18 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:40.195085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2b7ce-affe-4662-b3f3-271b00b34c18 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:40.196552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 167/335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:40.202385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:40.202853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:40.202945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:40.231896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:40.233110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:40.324181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 169/336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:40.653869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2b7ce-affe-4662-b3f3-271b00b34c18 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:40.791693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2b7ce-affe-4662-b3f3-271b00b34c18 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:40.836787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:40.837429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:40.838251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:40.838568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:40.859892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-128f28f4-a420-4022-820a-e4d78692372c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:40.867364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:40.868375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:40.869645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 168/337] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:40.874230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:40.944913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:40.954161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b033-4c8c-4323-9b1f-13b463eac158 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:40.954494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b033-4c8c-4323-9b1f-13b463eac158 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:40.954776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b033-4c8c-4323-9b1f-13b463eac158 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:40.956283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 169/338] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:40.967305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:40.967575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:40.967853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:40.997403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:41.000852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:41.011665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:41.026810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8de93-4b4d-47c8-8967-b3d2522b0e42 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:41.112523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 170/339] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:41.128539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:41.129193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.129489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.129754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.130487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:41.130832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:41.132285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b19bd-3b8d-43e6-8a11-5315e3f08ca3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:41.149822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 170/340] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:41.155813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:41.166832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9afef-b968-4d29-a084-29622b6bb3b9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.167119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9afef-b968-4d29-a084-29622b6bb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.167897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9afef-b968-4d29-a084-29622b6bb3b9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.169083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 171/341] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:41.180759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86dc0-c2b3-4e4c-86e0-f0949be696fa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.181039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86dc0-c2b3-4e4c-86e0-f0949be696fa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.181278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc86dc0-c2b3-4e4c-86e0-f0949be696fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.223538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0cc86dc0-c2b3-4e4c-86e0-f0949be696fa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:41.224693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0cc86dc0-c2b3-4e4c-86e0-f0949be696fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:41.300781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c9b72-e532-4a97-b2d8-8b5537249d15 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:41.323246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 172/342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:41.362224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:41.363357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.363855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.364371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.379129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:41.432362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb560-3a6c-41b2-a428-67c47556cf9f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.433024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb560-3a6c-41b2-a428-67c47556cf9f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.433509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfb560-3a6c-41b2-a428-67c47556cf9f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.435129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 173/343] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:41.440160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.440446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.441228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.462467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:41.464410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:41.479028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:41.480138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:41.583638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:41.600469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 171/344] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:41.606264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:41.648667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 174/345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:41.685061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:41.711585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:41.743465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f924ab9-e206-4b44-8560-8c3d72c7ece1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:41.804562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:41.805214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.805534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.805843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.811123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:41.812610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:41.814688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 172/346] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:41 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:41.819643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:41.885570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeacf53-764c-43fb-a2ae-683306674422 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:41.941856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:41.942533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:41.942815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:41.943092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:41.958388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:41.963818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:41.964711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:41.965610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 175/347] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:41.971445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56421f3-9192-4851-a8c8-0f28beb36491 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:41.987454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:41.991467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.033394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.034443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.035139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.036216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.049920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.052579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f97e24-23f3-456b-9427-1da9ed20b6a1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.056587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 173/348] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:41 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:42.059126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141632-1277-4324-ac8c-ce5ff26eae54 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.063697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141632-1277-4324-ac8c-ce5ff26eae54 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.064209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141632-1277-4324-ac8c-ce5ff26eae54 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.065679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 174/349] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:42.066615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8141632-1277-4324-ac8c-ce5ff26eae54 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.076032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.161214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8fee8-3409-4d60-a479-6ca66f4bd8cb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.203098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8141632-1277-4324-ac8c-ce5ff26eae54 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.204639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.205253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.206205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.206205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.206503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:42.206792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.208859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2854c5b-0c67-4f9d-8724-cbb95dad5527 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.218978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 176/350] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:42.220233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755107-2407-426d-b218-7708f02d82b7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.220522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755107-2407-426d-b218-7708f02d82b7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.220820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85755107-2407-426d-b218-7708f02d82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.243995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-85755107-2407-426d-b218-7708f02d82b7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:42.245435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-85755107-2407-426d-b218-7708f02d82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:42.248768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.249385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.249664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.249934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.250725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:42.251036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.252511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.259027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 175/351] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:42.262010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.313906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.343286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 177/352] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:42.344352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85755107-2407-426d-b218-7708f02d82b7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.404402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.426948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6210ad-e238-4927-86b9-67220596d636 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.471695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.472677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.472796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.473009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.486552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:42.493980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:42.494977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:42.495821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 176/353] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:42.496499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.505666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85755107-2407-426d-b218-7708f02d82b7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.550214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.550461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.550812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.551158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.562097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-dec8377d-0de9-4062-b639-669e0d6a260e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:42.567174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:42.568092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:42.568911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 178/354] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:42.570376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76847e-21f2-4f8c-9ccf-80dae4d4d67f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.572528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76847e-21f2-4f8c-9ccf-80dae4d4d67f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.572528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76847e-21f2-4f8c-9ccf-80dae4d4d67f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.574179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 179/355] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:42.575038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76847e-21f2-4f8c-9ccf-80dae4d4d67f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.620351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.626577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe9a2d-35f4-46de-8633-3bbfa5f1d948 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.669931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.670592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.670877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.671154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.681217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.690911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76847e-21f2-4f8c-9ccf-80dae4d4d67f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.742085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.742782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.743354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.743632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.745114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:42.745492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.747293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.757769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 180/356] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:42.761023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.772120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.773605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57d5b-39e5-45cf-ab7a-89836f2d4f64 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.888943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b803a35-cdde-453a-a8c7-415a13bd3663 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:42.893529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:42.901446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 177/357] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:42.902659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.902844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.910592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.925172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:42.928058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:42.940822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:42.941553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:42.942126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:42.942423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:42.943108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:42.943637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:42.945621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:42.956982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 181/358] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:42.957981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:42.984939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.010171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 178/359] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:43.015338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.088450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5685bd1-e6f1-47c9-a6d6-03dd0bc59c29 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.129182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.130233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.131769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.132322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.142540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d48c1d-51fe-4f91-92c6-ae37792a654e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.144987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:43.151077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:43.151986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:43.152832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:43.155259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 182/360] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:43.161042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.174314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.184477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.185150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.185433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.190480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.251027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.278321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3647c6-53df-4e99-b0bf-eb2b40211c3f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.278915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.280429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.327300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.327637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.327837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.328113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.329437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.330928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.332709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 183/361] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.333315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.422514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 179/362] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:43.422514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.452045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.454825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75a5b3-cb38-4457-95e0-913b9edb390e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.500129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.500996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.501252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.501563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] PATH_INFO: `/v3/users/19a471a8176b409b86c3d7b5b4355373` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.503239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] RBAC: Authorizing `identity:get_user(user_id=19a471a8176b409b86c3d7b5b4355373)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.504689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.506273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 184/363] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/users/19a471a8176b409b86c3d7b5b4355373 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.507089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.528938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.545572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4eabf-7e49-4c8e-84e8-299f06da276a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.595690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.596406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.596756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.596943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.597633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:43.597918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.599463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.608357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 180/364] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.608937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.640482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dec6e0-b458-4308-9885-6e2fbb237d6c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.688247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.689807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.690307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.691046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.705883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:43.711506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:43.712970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:43.713805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 185/365] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:43.714579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.731196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.747655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2953f-a2ff-4cc5-b7df-c9ac1d85af05 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.792971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.794946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.794946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.794946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.796193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.797673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.800509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 181/366] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.800509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.816195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:43.854531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772ff87-9c70-4a97-8889-24d86d4698c1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.907992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.909456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.910086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.910696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.912294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.914711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.917252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 186/367] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.918306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:43.935170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17e3aa-6a98-4b57-b7ef-7b59340f98ad None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:43.977721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:43.978672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:43.978672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:43.979011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:43.979836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] RBAC: Adding query filter params (role.id=b1efbfa1a5df4863bc966208c5270f82, user.id=19a471a8176b409b86c3d7b5b4355373, scope.system=all) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:43.980050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:43.981530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:43.993642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 182/368] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/role_assignments?user.id=19a471a8176b409b86c3d7b5b4355373&scope.system=all&role.id=b1efbfa1a5df4863bc966208c5270f82 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:43.998739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.042835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.057895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4a24-4ce8-4d5b-bd5c-d79f8eb86764 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.105948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.106609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.107268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.107545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.124215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.136749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15aa204-4e8c-4012-8503-6c47cc84f168 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.183814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.184426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.184706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.185079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.185853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:44.186163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.187593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.192460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.193920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.197777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 183/369] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.198713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.294895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.306339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 187/370] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:44.307047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.327693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b549ad1e-c0fd-400a-b26d-e7fceb58abb3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.362214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.378838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.380965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.380965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.380965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.381377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:44.381693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.385350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.389561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 184/371] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.390628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.446848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e07db6-fa56-4ce3-acb9-bf1cbfeeb90e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.500445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.501120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.501398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.501691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.502870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.504367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.506245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 188/372] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.507990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.524714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbd7b8-99e0-4857-a1d4-31c1cdc3068a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.569092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.569642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.569914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.570687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.584185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.654790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104c3b6-d9f1-4641-93c4-04e588dd29bc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.663369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.664846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.671472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.711842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.713280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.713729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.714145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.715572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.717309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182e4af-69ad-4f74-83cd-1a8645c6b009 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.719153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 189/373] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.720430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b89f-72bb-480a-b827-4d404a760bc7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.720884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b89f-72bb-480a-b827-4d404a760bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.721299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83b89f-72bb-480a-b827-4d404a760bc7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.723052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 190/374] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:44.723940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83b89f-72bb-480a-b827-4d404a760bc7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.800102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 185/375] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:44.800946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:44.850257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83b89f-72bb-480a-b827-4d404a760bc7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.853655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.891724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.892635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.893137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.894931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.894931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:44.894931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.896083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a7698-ade5-4022-86e3-9b97c79c5582 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.906682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 191/376] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.908077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.908361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.908635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.929795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606786e-3a8e-44c4-bc21-770ca8574fcc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:44.933960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:44.935302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:44.975441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:44.983016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:44.983991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:44.984438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:44.989381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:44.989381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:44.989381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:44.990531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 186/377] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:44.997029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.013576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 192/378] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:45.014526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.117784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052a071-4a6a-4ed9-b01e-bc9579862303 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.129222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:45.157867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.158788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.159260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.159684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] PATH_INFO: `/v3/projects/fd94b8179b9d4ba6806f99509678d7fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.161478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] RBAC: Authorizing `identity:get_project(project_id=fd94b8179b9d4ba6806f99509678d7fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.163614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.167075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 187/379] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:44 2022] GET /identity/v3/projects/fd94b8179b9d4ba6806f99509678d7fb => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:45.175615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.209357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c9836-226f-4485-a786-9683da1fe4aa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.302843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.303430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.303709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.303981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.304717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:45.305025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.306546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.318999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 193/380] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:45.325616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b07e7-dde6-409e-90e0-ebbf629b2a9d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.354280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.375503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.376377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.376815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.377225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] PATH_INFO: `/v3/groups/03adbe779ef948cea553668e86d16646` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.384023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] RBAC: Authorizing `identity:get_group(group_id=03adbe779ef948cea553668e86d16646)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.386272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.388978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 188/381] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/groups/03adbe779ef948cea553668e86d16646 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:45.395153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.425277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:45.504185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55a842-61ec-4223-9b5c-55dbc1cc44f5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.505376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:45.547357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.548659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.549112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.549525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.562692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:45.570006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:45.571131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:45.572661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:45.574620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 194/382] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:45.576783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2447f-c94f-49c8-a918-308083166dfc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.581403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.628727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.629264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.629538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.629803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.630560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] RBAC: Adding query filter params (group.id=03adbe779ef948cea553668e86d16646, role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=fd94b8179b9d4ba6806f99509678d7fb) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:45.630871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.632357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.648103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 189/383] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/role_assignments?group.id=03adbe779ef948cea553668e86d16646&scope.project.id=fd94b8179b9d4ba6806f99509678d7fb&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:45.707410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:45.716126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38de74b-9bf3-4db2-bcb0-96240880a81b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.760321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.760958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.761301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.761572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.762260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:45.762597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.764022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.774366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 195/384] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:45.779622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:45.920929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2969d-d14a-4b78-9f12-a00d85fe140c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:45.923296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:45.963781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:45.964416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:45.964695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:45.964968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:45.970801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:45.972261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:45.973861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 190/385] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:45.979430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:46.132392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d3edb-24e3-4adf-82ab-ee82c502c0c0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:46.170328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:46.171061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:46.171283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:46.171673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:46.172844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:46.174303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860eecfd-3356-4660-87eb-3a210108475b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:46.175945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 196/386] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:45 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:46.183452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:46.341976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4eb37-dcc6-4e01-aa46-7ad30ef09c3e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:46.392249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:46.392914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:46.393188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:46.393458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:46.395401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, group_id=0e3483caa2ca4bb8b69e1b4114320103, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:46.396855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:46.468721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd634-215b-4a99-8e33-ded93c9746bc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:46.469366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd634-215b-4a99-8e33-ded93c9746bc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:46.469366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cd634-215b-4a99-8e33-ded93c9746bc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:46.470620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 197/387] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:46.476539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:46.486659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:46.486659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:46.487066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 191/388] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 05:10:46 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:46.499229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:46.500580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:46.527269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:46.594005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 198/389] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:46.976564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:47.018916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:47.109828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b3fdf-32e7-4be9-add4-7c4952894cb7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:47.155661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:47.156388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.156875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.157331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.177294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-c1eaef6c-d92d-41bb-89c6-6628b1b93c3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:47.183261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:47.185515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:47.187556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 192/390] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:47.195107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:47.269747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388831ec-5fe7-4e02-8bcb-042feabc27cd None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.270051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388831ec-5fe7-4e02-8bcb-042feabc27cd None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.270308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388831ec-5fe7-4e02-8bcb-042feabc27cd None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.272447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 193/391] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:47.277175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.277458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.277746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.310048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:47.311305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:47.373109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6679f2b-ed6f-4e6d-8aee-3312e7d9ea6f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:47.423304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:47.423912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.424185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.424451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.425118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:47.425415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:47.426805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:47.443484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 199/392] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:47.451744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:47.463634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 194/393] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:47.502771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:47.621050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1135f95-7cf9-40ca-9310-2afc6fc25618 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:47.670944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:47.671671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.671945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.672215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.675191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:47.676645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba3bea-594b-4399-82dd-816315d06595 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:47.678244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 200/394] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:47 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:47.687501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:47.733083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:47.748708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd552353-2f11-4a3f-9b96-5ca00c5c74e8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.749140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd552353-2f11-4a3f-9b96-5ca00c5c74e8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.749422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd552353-2f11-4a3f-9b96-5ca00c5c74e8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.751942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 201/395] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:47.764816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.765093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.765369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.800903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:47.802098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:47.861134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13948c6d-d98d-46bc-a477-634730b8eec8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:47.892804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 202/396] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:47.893582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:47.904780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:47.906128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.906776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.907341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.908891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:47.911323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d95d337-2e5a-4c71-b655-53c87d7c9efe None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:47.912457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:47.915165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 195/397] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:47 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:47.916526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac6a6d-9a16-4712-9842-67a600b0e2b5 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:47.916973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac6a6d-9a16-4712-9842-67a600b0e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:47.917408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac6a6d-9a16-4712-9842-67a600b0e2b5 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:47.918775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 196/398] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:47.923595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac6a6d-9a16-4712-9842-67a600b0e2b5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.003295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15042def-1222-4bbf-b080-08d2626b7d23 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.023017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.035024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac6a6d-9a16-4712-9842-67a600b0e2b5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.040039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.040996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.041530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.041987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.053191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b1c50784-ad91-48cf-9ff2-e323e2f9fee5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:48.059882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:48.061635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:48.062481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:48.068881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 203/399] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:48.070507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.071055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.071652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.074042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.074736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.075010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.075286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.077808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] RBAC: Authorizing `identity:create_grant(project_id=026e5f20bad7459e8c9980fdb84e4d41, group_id=0e3483caa2ca4bb8b69e1b4114320103, role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.080800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.094501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:48.095709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:48.169047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 197/400] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 05:10:47 2022] PUT /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/119f085c1d714b95aa1674da1868b4a6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:48.169047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.172493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 204/401] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:48.174566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.186816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.301425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451232-f367-4356-9d86-a0d71867fa56 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.318955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e9b95d-6824-45b9-bddf-b09f8bce8a6a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.339269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.339884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.340258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.340537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.341412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:48.341721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.343139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.352425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 198/402] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:48.358782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.365089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.365736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.366016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.366287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.376065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.382512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:48.389680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:48.390681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:48.391767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:48.392433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 205/403] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:48.397592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.469663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.475710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8c3f-8c47-4a27-b6e7-de130696eef9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.508571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac6984-eaf3-44cd-9f59-0dcf537b44d9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.512599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.513280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.513564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.513838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.514990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.516818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.522538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 199/404] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:48.524498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.545796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.546429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.546729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.547001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.547676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:48.547977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.549457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.560288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 206/405] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:48.566443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.615702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.644981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3c13a-9770-4c31-b95b-c7b5a5543a2f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.681050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.681717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.681999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.682165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef033f4c-1dcf-4527-be23-a6b6619146b6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.682331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.683684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.685318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.690584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 200/406] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:48.691578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.709820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.718789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.719538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.719874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.720098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.721361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.722975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.725050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 207/407] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:48.730105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.805764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ad4be-eaaa-4a26-a1be-b16f5c450890 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.843559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d423fe-e54a-4e56-91fc-b78dbeb0bc29 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:48.871242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.871996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.873141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.873141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.880514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:48.881179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:48.882489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:48.882489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:48.883646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, group_id=0e3483caa2ca4bb8b69e1b4114320103, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:48.885200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:48.902433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:48.905188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:48.912820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 201/408] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:48.915113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:48.934495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:48.969220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 208/409] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 05:10:48 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:48.970605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.098276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73efcff-b73a-48a8-9707-e0569a1a7ef3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.123165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:49.144996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.145648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.145928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.146194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.147854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.149329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16d968-9c60-407e-b0f9-134fa6b3e688 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.152443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 202/410] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:49.156281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f91-47d5-47fb-8004-5a90da23624f None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.156281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f91-47d5-47fb-8004-5a90da23624f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.156780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf842f91-47d5-47fb-8004-5a90da23624f None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.158199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 203/411] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:49.159198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf842f91-47d5-47fb-8004-5a90da23624f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.167874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1a390-df0e-446a-913c-cb0405a7334e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.222515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.222515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.222515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.222515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.222906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:49.222906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.222906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33c5c7-9279-409d-aa27-eb9c62830d70 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.227000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 209/412] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:49.228275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.228555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.228824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.251742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:49.253122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:49.275751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:49.308051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf842f91-47d5-47fb-8004-5a90da23624f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.363925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 210/413] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:49.364971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.366216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.367175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.367665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.368131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.370712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] RBAC: Authorizing `identity:create_grant(project_id=862b6e5998d24e8e86fa766fd8ead0fb, group_id=0e3483caa2ca4bb8b69e1b4114320103, role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.372386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.418273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:49.434377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 204/414] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 05:10:49 2022] PUT /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb/groups/0e3483caa2ca4bb8b69e1b4114320103/roles/119f085c1d714b95aa1674da1868b4a6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:49.474171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4bc4f-e1aa-4521-9f55-eba274ba7004 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.512315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.513002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.513268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.513485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] PATH_INFO: `/v3/groups/03adbe779ef948cea553668e86d16646` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.514716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] RBAC: Authorizing `identity:get_group(group_id=03adbe779ef948cea553668e86d16646)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.516482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.518430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 211/415] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:49 2022] GET /identity/v3/groups/03adbe779ef948cea553668e86d16646 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:49.523625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.664104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9715c4c-d35f-45cb-b6c2-0cc31f2bf093 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.699331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.700460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.700963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.701447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] PATH_INFO: `/v3/projects/fd94b8179b9d4ba6806f99509678d7fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.702818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] RBAC: Authorizing `identity:get_project(project_id=fd94b8179b9d4ba6806f99509678d7fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.704471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.706630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 205/416] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:49 2022] GET /identity/v3/projects/fd94b8179b9d4ba6806f99509678d7fb => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:49.712608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.727095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:49.731499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:49.856032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325e96-b4d6-4162-bb43-eedac6a0907b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.868184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a0601-c4df-45b2-b1fb-a5aaf55c4023 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:49.901121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:49.924137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.925672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.926602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.926999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.934774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:49.937351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:49.937754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:49.938130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] PATH_INFO: `/v3/projects/fd94b8179b9d4ba6806f99509678d7fb/groups/03adbe779ef948cea553668e86d16646/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:49.942504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd94b8179b9d4ba6806f99509678d7fb, group_id=03adbe779ef948cea553668e86d16646, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:49.944168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:49.951886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:49.957503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:49.958657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:49.959685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:49.961711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 206/417] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:49.967298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:50.017303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 212/418] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 05:10:49 2022] PUT /identity/v3/projects/fd94b8179b9d4ba6806f99509678d7fb/groups/03adbe779ef948cea553668e86d16646/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:10:50.057734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:50.101397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fcdc1c-3b17-4f81-be89-6fc4484fb889 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:50.151198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:50.152098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.152522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.152961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.153792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:50.154251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:50.155879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:50.166130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 207/419] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:50.173036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:50.306672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3c780-34df-4542-a2b9-bc3b7de06e05 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:50.346360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:50.347341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.347851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.348345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.415889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:50.443098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:50.444611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc779ba-3d9f-453f-9433-0f23be54ce35 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:50.571237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 213/420] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:50.577560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:50.582676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa166d2-d9dc-401b-8b94-49380176818b None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.582965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa166d2-d9dc-401b-8b94-49380176818b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.583320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa166d2-d9dc-401b-8b94-49380176818b None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.584881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 214/421] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:50.592092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.592372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.592648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.622596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:50.623787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:50.750102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 215/422] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:50.762454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d616-fcd4-462e-b11c-8b93d44158f3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:50.802802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:50.815334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:50.818453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.818453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.818453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.819017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:50.820538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:50.823106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 208/423] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:50 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:50.828323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:50.842180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:50.947652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4b9c3-ad51-4328-9915-825480bdce73 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:50.984972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:50.985806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:50.986152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:50.986446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:50.987589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:50.989031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:50.990929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 216/424] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:50 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:50.997285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.075496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.107863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d51cc3-5c4d-44d7-9a79-10383e52e612 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:51.156123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:51.156728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.157006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.157276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.158005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:51.158310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:51.159780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed41f2-0552-4086-a11c-8311727dbc0c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:51.177588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 209/425] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:50 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:51.196998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:51.217970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8a1c-c178-4c78-afa1-ac225a5fe715 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.218315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8a1c-c178-4c78-afa1-ac225a5fe715 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.218550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8a1c-c178-4c78-afa1-ac225a5fe715 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.219906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 210/426] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:10:51.225377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.225647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.225920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.237118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66e238-53b2-45ab-a139-5dd807c62980 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:51.257472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:51.258837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:51.314062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:51.314786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.315062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.315331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.335419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:51.340233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:51.341126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:51.342366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 217/427] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:51.348028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.374780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:51.388281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 211/428] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 01 05:10:51.456270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7ab0b-7230-4661-b6d7-ff84c08c3598 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:51.491280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:51.491880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.492153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.492421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.493085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:51.494010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:51.495490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:51.500850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:51.504177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 218/429] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:51.509745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.627719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696343d9-5cf4-4895-8f0f-95ee3d200768 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:51.673740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:51.674721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.675139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.675564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.681506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.756652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:51.772000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:51.775058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:51.817933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d54ce-d951-42ec-ac2e-5d57200fa8e3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:51.860585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:51.861457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:51.861950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:51.862369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:51.874018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:51.881060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:51.882176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 05:10:51.883295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:51.885563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 219/430] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:51.894994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.920423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 212/431] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:51.926624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:51.978723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.035489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.036575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a24ee4-99d1-43b9-9366-f23b647b7bf7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.049402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d11f-278e-42d6-b236-c5da013d31f7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.077199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.077804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.078089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.078344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.079267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] RBAC: Adding query filter params (name=member) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:52.079372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.080670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfdf899-ddd9-4c5e-83b1-92ddd6ff3912 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.087008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.088106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.088384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.088654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] PATH_INFO: `/v3/projects/026e5f20bad7459e8c9980fdb84e4d41` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.094260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 220/432] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.094519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] RBAC: Authorizing `identity:get_project(project_id=026e5f20bad7459e8c9980fdb84e4d41)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.094642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e9b7-aabb-49a5-bb23-ff27b67e211e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.094642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e9b7-aabb-49a5-bb23-ff27b67e211e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.094642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2e9b7-aabb-49a5-bb23-ff27b67e211e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.094825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1567a18-a989-4f19-a2bd-96fe3ad66c2b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.096415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 221/433] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:52.097197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2e9b7-aabb-49a5-bb23-ff27b67e211e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.097788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 213/434] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:51 2022] GET /identity/v3/projects/026e5f20bad7459e8c9980fdb84e4d41 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:52.099040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.099325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.099599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.118284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:52.119542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:52.202635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 214/435] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:52.203041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2e9b7-aabb-49a5-bb23-ff27b67e211e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.203791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.224694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.245299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.245909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.246190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.246552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.310754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.330040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba100e-f8f2-4661-b02f-f94890a9619b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.331963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.334351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020d074-b8ab-4ea6-aef3-c71d18f6cbb1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.375366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.376377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.377040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.377553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.379604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.381204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.383385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 215/436] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.390876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.460983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 222/437] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:10:52.462174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d56c7-6067-4a83-ab2c-30b274a846ee None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.462469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d56c7-6067-4a83-ab2c-30b274a846ee None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.462742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d56c7-6067-4a83-ab2c-30b274a846ee None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.464109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 223/438] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:52.464640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d56c7-6067-4a83-ab2c-30b274a846ee None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.483586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.505675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532072b6-1f73-4f80-bdb3-9507cb39ed4b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.539120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.539727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.540003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.540297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.541022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=026e5f20bad7459e8c9980fdb84e4d41) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:52.541333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.542811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.548461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 216/439] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=026e5f20bad7459e8c9980fdb84e4d41&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.549519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.570313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.576550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d56c7-6067-4a83-ab2c-30b274a846ee None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.614676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.615315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.615591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.615865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.628177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c2bd5f01-c41f-44ea-9bb0-f789c6348999 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:52.634815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:52.635661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:52.637126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 224/440] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:52.637800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.637800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.637935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.658437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:52.659676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:52.671445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482995c-71bf-4b71-b628-2cff91e1f4f7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.714723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.714723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.714987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.715207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.716283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.717741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.719400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 217/441] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.720554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.740448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 225/442] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:52.741233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.753521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.819345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:52.836383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f2c34-882c-4afd-b07c-212c6e58577d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.879182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.880211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.880768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.881051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.881721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:52.882027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.882491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf27f6-bbfe-4ca8-9f8e-fdd46a6b09bc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:52.883509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.896535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 218/443] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.902388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:52.924083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:52.924715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:52.925001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:52.925256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:52.926661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:52.928112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:52.930307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 226/444] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:52.939571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.030600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da375a6-2ecd-4093-996f-9c9624c0910d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.032251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.055338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b981ab-eac5-4c9f-8ffb-90313e992e61 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.063912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.064515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.064789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.065055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.082027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.090680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.091270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.091921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.091921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.092499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=2b04b45f17f1428facf4bd2291e8bcb3, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:53.092803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.094223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.099831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 227/445] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=2b04b45f17f1428facf4bd2291e8bcb3 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:53.111773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.137062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.138646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.225527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c05c89-139f-463b-84f4-f830290337a5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.260036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 219/446] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:53.260036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.269720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.270370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.270964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.271299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.282098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.286205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:53.291430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:53.292591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:53.293405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 228/447] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:53.298458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.318598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.374914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d300-4b03-4194-8ae8-e858f36f594e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.415711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.416371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.416647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.416922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] PATH_INFO: `/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.418129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] RBAC: Authorizing `identity:get_project(project_id=862b6e5998d24e8e86fa766fd8ead0fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.419664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.421478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 220/448] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/projects/862b6e5998d24e8e86fa766fd8ead0fb => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:53.422830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1e8f3-b04e-457c-ae06-97de460f60a0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.428312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.460472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.461089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.461366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.461634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.462866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:53.463168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.464536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.472652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 229/449] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:53.478831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.531002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1c72c-3431-4e02-8f34-3839cf8c663e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.537570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.547465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.564203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.564839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.565118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.565390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] PATH_INFO: `/v3/groups/0e3483caa2ca4bb8b69e1b4114320103` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.566586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] RBAC: Authorizing `identity:get_group(group_id=0e3483caa2ca4bb8b69e1b4114320103)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.568037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.569607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 221/450] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/groups/0e3483caa2ca4bb8b69e1b4114320103 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:53.576151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.576519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cce31-8f81-4ad0-a27d-97f2b0786a02 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.610322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.610954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.611235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.611506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.676612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8afc2-3ff4-4171-b0db-3c8528b6a0e7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.677494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.679012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.709744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.710346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.710694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.710917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.711642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] RBAC: Adding query filter params (group.id=0e3483caa2ca4bb8b69e1b4114320103, role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:53.712548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.714069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.719729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 222/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/role_assignments?group.id=0e3483caa2ca4bb8b69e1b4114320103&scope.project.id=862b6e5998d24e8e86fa766fd8ead0fb&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:53.738398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.783505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 230/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:10:53.789415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:53.840967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:53.892471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa67a6-fa91-435b-881e-5a23d1ac0ba7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:53.925445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:53.926078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:53.926362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:53.926667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] PATH_INFO: `/v3/projects/fd94b8179b9d4ba6806f99509678d7fb` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:53.927793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] RBAC: Authorizing `identity:get_project(project_id=fd94b8179b9d4ba6806f99509678d7fb)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:53.929257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:53.931449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 223/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/projects/fd94b8179b9d4ba6806f99509678d7fb => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:10:53.937129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:54.037616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaaa373-e1a1-481a-b7b0-3ff7d24ed085 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:54.071016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:54.072172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:54.072446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:54.072714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] PATH_INFO: `/v3/groups/03adbe779ef948cea553668e86d16646` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:54.073811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] RBAC: Authorizing `identity:get_group(group_id=03adbe779ef948cea553668e86d16646)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:54.075291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fe5cf-0419-4579-b40a-81b54513bf25 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:54.076840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 231/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 05:10:53 2022] GET /identity/v3/groups/03adbe779ef948cea553668e86d16646 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:54.082660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:54.177353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55527ac-7d58-48d1-8bdd-24353d8c85dc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:54.192828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:54.209743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:54.210328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:54.210630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:54.210902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:54.211620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] RBAC: Adding query filter params (group.id=03adbe779ef948cea553668e86d16646, role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=fd94b8179b9d4ba6806f99509678d7fb) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:54.211930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:54.213356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee16577-2cb6-49e2-af44-7fd5e585144a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:54.218818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 224/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 05:10:54 2022] GET /identity/v3/role_assignments?group.id=03adbe779ef948cea553668e86d16646&scope.project.id=fd94b8179b9d4ba6806f99509678d7fb&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:56.459660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a09da-172a-4216-b8a4-fca7de9fbb13 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.460674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a09da-172a-4216-b8a4-fca7de9fbb13 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.461157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a09da-172a-4216-b8a4-fca7de9fbb13 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.463078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 232/456] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:56.468206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1742d-c2e5-4427-aa62-a6454a10e56d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.469096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1742d-c2e5-4427-aa62-a6454a10e56d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.469573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1742d-c2e5-4427-aa62-a6454a10e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.488263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-18f1742d-c2e5-4427-aa62-a6454a10e56d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:56.489778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-18f1742d-c2e5-4427-aa62-a6454a10e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:56.561529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 225/457] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:56.649610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da11c1-959c-483a-a3b1-2939b13ad685 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.649880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da11c1-959c-483a-a3b1-2939b13ad685 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.650381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da11c1-959c-483a-a3b1-2939b13ad685 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.652160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 233/458] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:56.657432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30def53-be6d-4ff1-b5c3-b5c26ff43b20 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.657968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30def53-be6d-4ff1-b5c3-b5c26ff43b20 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.658473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30def53-be6d-4ff1-b5c3-b5c26ff43b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.687354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d30def53-be6d-4ff1-b5c3-b5c26ff43b20 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:56.688887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d30def53-be6d-4ff1-b5c3-b5c26ff43b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:56.776920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 226/459] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:56.833552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426d5f4-f543-4514-b17d-2ada1c738064 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.834151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426d5f4-f543-4514-b17d-2ada1c738064 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.834683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426d5f4-f543-4514-b17d-2ada1c738064 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.836436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 234/460] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:56.841877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:56.842442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:56.842924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:56.862952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:56.864463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:56.881896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426d5f4-f543-4514-b17d-2ada1c738064 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:56.902684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:56.919174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:56.965267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 227/461] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:57.035163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426d5f4-f543-4514-b17d-2ada1c738064 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.079091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.079788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.080064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.080340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.093597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.095082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:10:57.113480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:57.114521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:10:57.115406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:10:57.115406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.115406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:57.115406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 235/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:57.122324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.146436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.245713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:57.283102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2ace-dc39-46e7-ae7b-167c3f8f6d6d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.306699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a761f8-432a-4067-bb62-65ea57e8b384 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.357739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.359274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.359733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.362106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.375953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.376912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.377781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.378250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.379261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:57.379436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.379938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.380893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.381717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:10:57.395913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:57.396858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:10:57.397714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:10:57.397714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.397714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:57.398496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 236/463] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:57.399533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.402915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 228/464] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:57.403978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.435469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:57.515876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609786d-b19f-4436-855f-e8c0a82ae725 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.527830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68346efe-fec9-45de-aa60-619c84196cb3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.552847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.553457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.553734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.554003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.554663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:57.554956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.556969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.565931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.566817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.567243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.567663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.572336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 229/465] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:57.572336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.573457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:57.584550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.586020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4cd74a6c-3695-4813-a979-870e18f5e485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:10:57.599865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:57.610153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 237/466] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d234015-8d47-48b8-a3d4-9f2de6ff43a4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d234015-8d47-48b8-a3d4-9f2de6ff43a4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d234015-8d47-48b8-a3d4-9f2de6ff43a4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 238/467] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:57.611345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d234015-8d47-48b8-a3d4-9f2de6ff43a4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.674298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:57.683902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620fe4d-3556-4f1a-a1be-352a8f22dc75 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.719153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.719758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.720039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.720309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.720953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.721988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d234015-8d47-48b8-a3d4-9f2de6ff43a4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:57.722439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.758084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:57.758868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.759184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.759439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.760083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:57.761517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4f493-d227-4eaf-b513-bd21b197aaff None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:57.805013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 230/468] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:57 2022] POST /identity/v3/users => generated 246 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:57.805885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:57.865798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 239/469] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:57 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:57.868386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:57.869177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:57.869677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:57.895696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:57.897601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:57.949613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:57.997460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 240/470] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:10:58.006675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7ad15-145f-4d57-9b21-7e69c8e2b888 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.048842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.049258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.049535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.049804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.050493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:58.050797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.052202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e058ea5-ad1d-4554-9700-572895941373 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.066843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 231/471] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:58.071564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.160700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a85c-8f24-4d33-8324-b1a26bf5c107 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.160975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a85c-8f24-4d33-8324-b1a26bf5c107 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.161172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a85c-8f24-4d33-8324-b1a26bf5c107 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.162829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 232/472] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:10:58.167861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.168147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.168427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.186793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:58.187948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90822fe6-f236-484b-b33a-8c68a22fd3c2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.195418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:10:58.197166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:10:58.245953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.246669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.246947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.247219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.247888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.249370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.301509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 233/473] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:10:58.322375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.377088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 241/474] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:58 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:58.435741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bae20-7a6b-4cc3-bd5e-c9e11d34d928 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.472532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.473278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.473555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.473827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.477214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:58.487040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.488560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:10:58.506784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:58.507787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:10:58.509329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:10:58.509329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:10:58.509329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.509329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 234/475] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:58.514797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.566856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:58.626383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.646373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71db0c-37b4-4dfc-ad4f-9a1196d9cf3a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.693808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.694671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.694955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.695230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.696979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:58.697299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.698863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.715782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 242/476] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:10:58.722898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.756325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54efc7f-c2c0-42a6-9929-275dbc2b59d3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.792684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.793535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.793983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.794432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.795256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.796994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.828558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:58.838890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.sql.core [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:10:58.838890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 05:10:58.838890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [parameters: {'id': '26fe68f124244689b39faffc431abffe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 05:10:58.838890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81568) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:10:58.840894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880044e-29a4-4bce-bcac-aed3092112f8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application result.read() Sep 01 05:10:58.863240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:10:58.864395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application yield resource Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise self.value Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application session.commit() Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:10:58.865559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:10:58.866608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:10:58.867586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application result.read() Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:10:58.869048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [parameters: {'id': '26fe68f124244689b39faffc431abffe', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:58.869848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 05:10:58.870697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:58.873351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 235/477] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 05:10:58 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:58.879224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:58.905180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:58.905807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:58.906088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:58.906357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:58.907207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:58.909289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:58.954035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:10:58.993410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bde57-053f-4b07-bf23-e075c0a71c78 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:59.028826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 243/478] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:10:58 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:10:59.036669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:59.037322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:59.037824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:59.038106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:59.050864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-e30d29eb-68da-41dd-93cc-fc6525707dfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:10:59.057938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:10:59.059040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 05:10:59.059855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:10:59.062908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 236/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:10:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:10:59.068735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:10:59.205519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9887d0-bd7b-447c-aba0-ced4c0b35afd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:10:59.247861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:10:59.248471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:10:59.248750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:10:59.249026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:10:59.249696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:10:59.250000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:10:59.251493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c78f-85ad-4a61-9f26-94570c7bef2b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:10:59.260039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 244/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:10:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:00.272881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9adfe-f060-481a-b510-a3dd3b3174d4 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:00.273862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9adfe-f060-481a-b510-a3dd3b3174d4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:00.274363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9adfe-f060-481a-b510-a3dd3b3174d4 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:00.275939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 237/481] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:00.281191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24f6e-bda7-408e-b173-24a26e5f01a3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:00.281992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24f6e-bda7-408e-b173-24a26e5f01a3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:00.282512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24f6e-bda7-408e-b173-24a26e5f01a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:00.301185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:00.308212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97f24f6e-bda7-408e-b173-24a26e5f01a3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:00.309397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97f24f6e-bda7-408e-b173-24a26e5f01a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:00.392464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 245/482] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:00.418610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82900ea-6ffb-4473-9415-9492d7391fda None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:00.418939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82900ea-6ffb-4473-9415-9492d7391fda None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:00.419158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82900ea-6ffb-4473-9415-9492d7391fda None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:00.420555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 238/483] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:00.425968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:00.426436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:00.426871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:00.454652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:00.456071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:00.550332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 246/484] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:00.775395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82900ea-6ffb-4473-9415-9492d7391fda None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:00.908181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82900ea-6ffb-4473-9415-9492d7391fda None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:00.922743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:00.951696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:00.952386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:00.955459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:00.955749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:00.966273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:00.970329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:00.975304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:00.976215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:00.977060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:00.982821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 239/485] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:00.983704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.037377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.077033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a5e3-3ca0-4ef4-8932-895588fc6f65 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.114671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.115805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.116295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.116814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.124346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4418a3b-df84-4a99-a96a-59c8e48f12d9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.130043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:01.135796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:01.136721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:01.137547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:01.145838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 247/486] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:01.147627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.219211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.219753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.220032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.220307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.221036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:01.221342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.222862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.241762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 240/487] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:01.247483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.264793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacd77c-5e18-4e5b-872b-788bdd7954cb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.297091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.301836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.313968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.315073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.315500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.316081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.317216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:01.317670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.319230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940da75-334a-4628-9ddd-0a1b6e751740 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.336172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 248/488] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:01.336172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe57-ebe1-42c1-abf5-be4dc11f63f9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.344260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe57-ebe1-42c1-abf5-be4dc11f63f9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.344260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bfe57-ebe1-42c1-abf5-be4dc11f63f9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.344260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 249/489] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:01.344260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfe57-ebe1-42c1-abf5-be4dc11f63f9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.385827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad014-e0d4-4e23-8186-f4ef100d31b2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.429015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.429638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.429916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.430208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.440797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.442258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.454137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bfe57-ebe1-42c1-abf5-be4dc11f63f9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-bf706e70-4b5d-4dc7-86c3-f8df8a0b43d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:01.458159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:01.459101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:01.459975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:01.459975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:01.459975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:01.459975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 241/490] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:01.460905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.461186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.461971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.482286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:01.483657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:01.496325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.497037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.497320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.497597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.509027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.509901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-84ff9e14-8970-45c2-8c3b-bc9f99540793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:01.526034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:01.527114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:01.527979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:01.527979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:01.527979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:01.527979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 250/491] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:01.528708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d77a9-d4cb-4b46-9f77-ad5151758320 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.528991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d77a9-d4cb-4b46-9f77-ad5151758320 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.529261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d77a9-d4cb-4b46-9f77-ad5151758320 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.530675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 251/492] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:01.531196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d77a9-d4cb-4b46-9f77-ad5151758320 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.562121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 242/493] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:01.563250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.590034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.592047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.660849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d77a9-d4cb-4b46-9f77-ad5151758320 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.683670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325d6ca-19ab-4846-a211-87603ad7d150 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.695133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.695797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.696175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.696555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.697204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:01.697498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.698910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf219257-ca6f-4c8e-9899-ffabc79af988 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.710842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 252/494] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:01.712602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.712885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.713158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.723147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.723753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.724027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.724294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.724926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:01.725230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.726632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69904b3a-7cb6-45bb-bc23-d8df69c36902 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.732368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:01.733506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:01.739077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 243/495] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:01.740525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76738717-84ba-4941-afc4-18256c761e7b None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.740809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76738717-84ba-4941-afc4-18256c761e7b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.741076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76738717-84ba-4941-afc4-18256c761e7b None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.742358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 244/496] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:01.743021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76738717-84ba-4941-afc4-18256c761e7b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.810130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 253/497] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:01.811198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:01.841110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.842078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:01.868651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76738717-84ba-4941-afc4-18256c761e7b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.903860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.904497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.904785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.905059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.912900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.913081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2f210-8221-4c0c-85e0-e0f8342a1259 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:01.914371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-6345aea0-49dc-4c7b-a58e-507dca21527e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:01.927003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:01.927971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 245/498] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd31c-f906-446c-8908-b0448b492487 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.934022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd31c-f906-446c-8908-b0448b492487 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.934787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dd31c-f906-446c-8908-b0448b492487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.948706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:01.948884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-581dd31c-f906-446c-8908-b0448b492487 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:01.949336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:01.949613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:01.949885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:01.950048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-581dd31c-f906-446c-8908-b0448b492487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:01.958961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:01.960762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:01.974394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:01.975360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 254/499] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:01.976281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.042440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 246/500] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:02.042440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dd31c-f906-446c-8908-b0448b492487 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.083261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.093582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17439240-fbdc-4d66-a71a-d8c828c87684 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.111653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.131953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.132589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.132865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.133132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.146175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dd31c-f906-446c-8908-b0448b492487 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.146325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:02.151999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:02.152904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:02.153739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:02.157925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 255/501] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:02.158794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.183213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.184047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.184318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.184676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.185497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:02.185891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.187461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.223426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 247/502] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:02.224363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.263276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc5db-4f86-43c1-a0f2-19671a9bc637 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.297584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.298631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.298916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.299184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.300247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:02.300554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.302585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.324380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3728c-2636-4f81-85df-a8a01ca955c4 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.341320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 256/503] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:02.342156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.348356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.353719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.358626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.359257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.360199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.360199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.370496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:02.375247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:02.376209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:02.377061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 248/504] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:02.377777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.445199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6e253-4f33-41dd-b36e-6e1439f1ddbe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.473694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d50d5-1713-4b34-a908-63006a08006f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.478531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.479125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.479397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.479662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.480314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:02.480612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.481975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.489609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 257/505] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:02.490389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.508429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.509030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.509302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.509573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.558236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.559770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.575020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.576541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.587729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fd602-b3d8-4ff6-a9f8-dde79d79e1c6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.619960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.620586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.620922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.621199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.634080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.637764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:02.651167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:02.652119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:02.652980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:02.652980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:02.652980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:02.652980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 258/506] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:02.653205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.678298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 249/507] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:02.679249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.755691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c01616-a9f2-4d9b-a16c-7e35c006d16c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.760548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.777450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111bdf1-fe02-45ee-937e-f9989c631d8d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.778116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.791758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.792356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.792632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.792898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.812068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.812724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.813010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.813285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.813956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:02.814260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.815777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.828428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 250/508] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:02.828428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.859484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.860938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.923792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6b97-58b1-4747-9b31-8a14411e7c53 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:02.955990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 259/509] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:02.956842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:02.960781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:02.961407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:02.961686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:02.961956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:02.972570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:02.974022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:02.979178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.980293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:02.987965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:02.995286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:02.996157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:02.996157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:02.996157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:02.996157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 251/510] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:02.996157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.054141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4665c0-b770-4793-80a2-26df3151c584 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.087032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96150986-7be0-4da0-a1f4-6e8cd4d13e05 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.087393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.087987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.088262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.088533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.089176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:03.089476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.090886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.102111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 260/511] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:03.102934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.120623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.121255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.121539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.121809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.132228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.134267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:03.148377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:03.149319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:03.155025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:03.155025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.155025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:03.155025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 252/512] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:03.155025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.184806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.202188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.204373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de668-2eb9-4d16-a7db-ef2cf505ec62 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.237123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.237771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.238052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.238324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.249429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.250517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a031207-d7ec-40cd-a52e-cef2d31b8938 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.250932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:03.266098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:03.267228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:03.268270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:03.268270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.268270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:03.268270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 261/513] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:03.268270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.288508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.289167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.289950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.290223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.300526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.301997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:03.316417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:03.317511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:03.318400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:03.318400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:03.318400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:03.318400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 253/514] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:03.318400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.366974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbead4ab-cff0-48a9-9f9c-701dbf6ef9bf None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.399724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.400319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.400590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.400860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.401490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:03.401786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.403177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.404535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.416074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 262/515] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:03.416947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.418976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a2ad5c-c14c-4163-8984-77c23adfee1a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.421030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.453763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.454356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.454664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.454973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.455887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.457863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.513422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131da4-61b1-43d5-b7a8-ff791d252d29 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.542537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 254/516] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:11:03 2022] POST /identity/v3/users => generated 247 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:03.543353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.558900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.559529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.559824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.560081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.560724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:03.561023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.562531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.579452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 263/517] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:03.579452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.651422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.654037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f096c-be51-42b6-9362-c7fc77f77e06 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.694269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.695629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.695629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.695629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.696099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:03.696405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.697795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.702943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a370e-330b-4c2a-954e-ae9151c85f44 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.711175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 255/518] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:03.712354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.736090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.746396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.747038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.751904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.751904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.751904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:03.751904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.751904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.763022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 264/519] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:03.766455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.819672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee19e1-c7c4-4ce0-8e47-6f5fd8a791ab None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.856244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.856978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.857340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.857527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.864651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.866052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.876232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491db18-4eeb-4968-bc6b-fd6d892529ae None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:03.878110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:03.879039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:03.879948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 256/520] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:03.880509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.895216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.916663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:03.917291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:03.917559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:03.917885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:03.926296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:03.927794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:03.939410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:03.940420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:03.941441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:03.943272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 265/521] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:03.948629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:03.968149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:03.993026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591313b-10cf-4d34-873f-e01fb4a2992b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.030707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.031335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.031610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.031878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.040594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.042149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.051333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2ac9e-df8a-4caf-a362-5e3543257265 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:04.053846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:04.055034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:04.055969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 257/522] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:04.056548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.094608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.095247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.095524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.095792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.104274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.105743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:04.117556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:04.118501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:04.119386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:04.123264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 266/523] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:04.123971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.161865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.167942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca02e70-6b8c-4350-8a51-fc6cb8dc5843 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.202692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.203309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.203606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.203858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.204939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.204939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.206334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.216478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.230856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 258/524] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.230856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.241122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e264b-af67-40ed-a161-7953eff5853e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.276906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.277509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.277784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.278057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.278765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.279067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.280509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.298156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 267/525] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.304173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.333499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b8a80-25b4-4c29-98c2-d2fa3df7521f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.368953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.369566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.369844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.370117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.370878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.371186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.372630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.391497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 259/526] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.392495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.409397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.421801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96ed34-efe2-453e-8817-cd43a9cbc209 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.450236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.454849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.455676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.456048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.456389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.457182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.457555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.459140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.475935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 268/527] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.475935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.501333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db6fa8-78af-4e14-ad1f-5e4db6377a45 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.539531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.540225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.540572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.540913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.580915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423be980-f313-43b5-a48c-f4370522b15c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.610525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.611997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.618786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.619592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.620179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.620522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.621308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.621677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.623306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.642813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 269/528] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.643612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.677028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.688167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.754897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 260/529] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:04.754897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.806269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61cd771-b467-4986-8672-dca1aa8dbfb4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.855308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.856462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.856853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.857231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.866352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e4413-bd4b-4abf-a85d-33e7c3078698 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:04.906212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:04.907073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:04.907452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:04.907830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:04.908670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=fc91470011c746488ea56ca5d9e7a8e4) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:04.909081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.910693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:04.926498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 261/530] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/role_assignments?user.id=fc91470011c746488ea56ca5d9e7a8e4&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:04.926498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:04.944192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:04.974446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:04.976154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.011532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.044355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4165d-a318-4f66-8064-12072efcd493 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.103323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.104344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.104829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.105302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.135345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.137208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.138759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 270/531] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:05.145264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:05.155564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:05.156719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:05.157827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:05.157827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:05.157827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:05.157827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 262/532] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:05.162623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.267777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ead791-d0a7-4742-853f-fb8b05a22152 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.273220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.288798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.308418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.309138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.309481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.309815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.321074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.322655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:05.339678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:05.341901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:05.344021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:05.344021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:05.344021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:05.344021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 271/533] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:05.348844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d760c-cc13-43c0-8c9c-1622e55edf01 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.350223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.424386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.425040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.425318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.425638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.426291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:05.426620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.428004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.464672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 263/534] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:05.469702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.491942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85629-b18a-40da-80e8-bf793540da2e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.526174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.526867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.527147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.527422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.528068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:05.528370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.529943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.542834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 272/535] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:05.549558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.583657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.649621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30d0ef-723d-4c67-9259-2e6a59d0fced None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.670198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc67c6-1fe1-47e4-a8dc-fcd451e2403c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.685244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.693525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.694629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.705894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.705894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.705894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.707288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.708066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.708780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.709129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.709466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.717752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.719513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-c94b68b9-0ccf-4555-b4b7-7e43558639d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:05.721813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:05.723866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 264/536] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f2a0-1182-4153-81b5-c67af7f804f8 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.725838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f2a0-1182-4153-81b5-c67af7f804f8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.727128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1f2a0-1182-4153-81b5-c67af7f804f8 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.731435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 265/537] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:05.731435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1f2a0-1182-4153-81b5-c67af7f804f8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7de070af-09f0-4031-8ee5-9724e6bed076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:05.738873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:05.740616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:05.742468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 273/538] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:05.747052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.747052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.747052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.763672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:05.765435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:05.850757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 274/539] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:05.851865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:05.861329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1f2a0-1182-4153-81b5-c67af7f804f8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:05.888790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:05.934653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:05.935188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:05.935461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:05.935730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:05.939061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:05.939373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:05.943520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:05.973828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 266/540] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:05.979243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.014022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183dab96-77f7-4439-b6ff-8eec57b146d6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.073175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:06.079017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.084059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.106010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 275/541] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:06.115818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.152896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89103f2-539e-417b-b82f-7f9c8f549254 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.211497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.212120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.212389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.212656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.213383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=b6cb1f27ea5c49c9a805a9cdd678bef6) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:06.213687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.215169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.221245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 267/542] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:05 2022] GET /identity/v3/role_assignments?user.id=b6cb1f27ea5c49c9a805a9cdd678bef6&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:06.222134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.238947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:06.260871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5cc66-59a5-42bf-8aea-48303fe5e4e1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.307468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.308386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.308809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.309256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.310132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=1d00e1ef6c1a4f63a9a4514aac586c67) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:06.310609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.312240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.326833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 276/543] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/role_assignments?user.id=1d00e1ef6c1a4f63a9a4514aac586c67&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:06.332666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d065af-1d5d-4231-8db6-dfd06f52904c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.349736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:06.374747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.375282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.375615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.375886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.389174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:06.395059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:06.396036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:06.397270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 268/544] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:06.402441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.519027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788195-d751-47e0-8cdd-08cf29875f83 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.557214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.558218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.558727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.559211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.560081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:06.560591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.562180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a3ed4-76ff-42e1-8eb7-794eed0d8b17 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.571515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 277/545] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:06.578460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.714282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a04b-81af-441f-b79e-2308e8862900 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.750036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.750699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.750976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.751244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.762246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ef2f-259e-4e74-a095-994bf2f5ac58 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.762636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.763083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ef2f-259e-4e74-a095-994bf2f5ac58 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.763551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227ef2f-259e-4e74-a095-994bf2f5ac58 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.764105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.770595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 278/546] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:06.776204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.776740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.777208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:06.779986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:06.780939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:06.782356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:06.782356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:06.782356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:06.782356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 269/547] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:06.790444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:06.806773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:06.817878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:06.827337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:06.827337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:06.892579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3f705-b4fd-490a-b64a-029d2d3203ca None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:06.929330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:06.930031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:06.930313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:06.930612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:06.931255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:06.931558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:06.932937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:06.945648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 270/548] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:06.950993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.015550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 279/549] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:07.041109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:07.056298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24818-ab14-445b-8baf-f1b4543a7208 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.096457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.097094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.097369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.097631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.109606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:07.111126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-ab86284c-fd6f-4d55-ade0-43ac1a97b914 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:07.126994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:07.127981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:07.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 271/550] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:07.135519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.224298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52d77-3df5-4486-af2a-cd18064736e5 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.224595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52d77-3df5-4486-af2a-cd18064736e5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.224763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c52d77-3df5-4486-af2a-cd18064736e5 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.226397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 272/551] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:07.231476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.231758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.232029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.267398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:07.268594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:07.277571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d693-dffc-4892-806c-d8b64c1d061c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.319353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.319963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.320235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.321032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.321762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:07.322061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:07.324920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:07.331057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:07.354176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 280/552] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:07.360938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.407747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 273/553] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 05:11:07.409416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.506841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:07.516894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dfaf4-a873-4fbc-9352-18fb864121a5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.523511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430b286-f7eb-4080-84e6-bf50ea84e5f7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.593003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.593615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.593891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.594160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.596486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.597117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.597393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.597657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.616341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:07.622664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:07.623591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:07.624513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:07.625168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 274/554] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:07.630427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.714676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:07.728832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:07.730305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:07.794334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec8e3e-91dd-48e2-8292-bd063198e1ce None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.829914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.830945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.830945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.831122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.831750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:07.832049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:07.833424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:07.836118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:07.843178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:07.844189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.881501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 281/556] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 05:11:07.882765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:07.953531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93b848-11e0-4fd3-8477-f62dc169a2ce None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:07.991024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:07.992060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:07.992535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:07.992971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:07.997647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.005498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:08.006388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952ddc9-11d8-4b0c-bdd5-53a7af33b285 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:08.010978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.017972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 276/557] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.018999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.056124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.057008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.057425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.058242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.073053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.075513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.079800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:08.096580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.097919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:08.099052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:08.099052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.099052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:08.101100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 282/558] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.102138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.131965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0441-83b5-479b-a6bd-ea3832826240 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.181511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.182436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.182862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.183322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.197983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.200025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:08.217522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.218549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:08.219527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:08.219527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.219527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:08.223276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 277/559] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.224100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.231094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84433b-280a-4348-bfe9-7b2c2f42d37e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.277820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.278832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.279311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.279818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.281215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:08.282032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.284282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.295930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 283/560] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:08.297419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.312321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.332637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00a7f1-886d-4670-a1ca-1dfb1659a7d3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.344545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.378379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.379495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.379949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.380750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.381876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:08.382328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.384449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.401087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 278/561] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:08.402666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.427863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9d8f1-5b5d-4307-86d3-364611a44c03 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.465069 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.465945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.466220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.466515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.467154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:08.467455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.468825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.482563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 284/562] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:08.482563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.521216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c67a-c1da-4ea7-bbdb-b9f3652f8543 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.555590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.556593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.557218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.557494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.557763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.570483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.570483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-8d321c8d-c0bd-40d1-9c6a-a625fa19f185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:08.583912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.584863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:08.585770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:08.585770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:08.585770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:08.585770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 279/563] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.586526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6e39-0c59-4234-a464-f2a3f324743f None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.586526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6e39-0c59-4234-a464-f2a3f324743f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.586767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e6e39-0c59-4234-a464-f2a3f324743f None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.588087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 280/564] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:08.588605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e6e39-0c59-4234-a464-f2a3f324743f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.599353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105cc91c-d7e9-462b-82fc-f218383cb662 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.616080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.639634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.640527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.640861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.641139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.649273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.650950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:08.663340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.665097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 285/565] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.666079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.701396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e6e39-0c59-4234-a464-f2a3f324743f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.734993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.735624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.735910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.736182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.743364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.744843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-c6038000-e4f5-4290-b02f-ddad30a326a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:08.755959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:08.757047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 281/566] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.758497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.759151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.768264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bb727-b5a8-4ec9-b887-a0d80581b5f4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.776890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:08.778067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:08.793974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.801286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.801893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.802166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e5933-bd58-4d6e-a254-61e5c0c16d06 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:08.822473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c0b9-ed36-40a0-aa4b-5d1832e4c4e4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.823038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c0b9-ed36-40a0-aa4b-5d1832e4c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.823038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3c0b9-ed36-40a0-aa4b-5d1832e4c4e4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.823038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 287/568] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:08.823038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3c0b9-ed36-40a0-aa4b-5d1832e4c4e4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.848514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 282/569] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:08.849611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:08.864214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:08.951572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6aef7-5aad-4d93-9ba7-d12bd077b93b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.979248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3c0b9-ed36-40a0-aa4b-5d1832e4c4e4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:08.986862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:08.987437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:08.987717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:08.987988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:08.988680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:08.988983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:08.990439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.001824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 283/570] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.007247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.016310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.016912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.017187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.017455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.018170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:09.018681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.019927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39859abb-7759-46ff-993e-394a46d8ab66 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.038728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 288/571] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.040705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.041057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.041398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.060077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:09.061378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:09.084189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.106066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.116143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ec32e-1396-4d7a-9d10-2c0adf8d519b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.130725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 289/572] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:09.131558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.166142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.166804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.167079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.167347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.175956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.177216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:09.190400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:09.191592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 284/573] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:09.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.243432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41b2c-07f5-4590-a728-fef68686c4d1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.282325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.283083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.283478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.283848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.284671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=01a4e7f4d1e54b17938c3514428a34b9) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:09.285079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.286629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.293502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 290/574] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/role_assignments?user.id=01a4e7f4d1e54b17938c3514428a34b9&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.294275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.314346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3dbac-ef1f-4583-b5d7-ff760b44a879 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.340697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.355219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.363322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.363971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.364255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.364527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.377050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.378697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:09.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:09.397823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:09.399533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:09.399533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:09.399533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:09.402862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 285/575] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:09.404044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.420605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c41e-b87a-4d71-8078-8a91e6cd0acd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.460443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.461680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.461956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.462223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.472601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:09.477792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:09.478794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:09.479673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.521248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab967ab-7bb8-44ac-9499-085baeb8a9a0 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.557112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.557722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.557997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.558270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.559038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:09.559340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.560787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.579175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 286/577] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.583350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.590489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.593567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86b8b2-6fa8-416d-b3a5-86311789c005 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.627771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.628421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.628641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.628910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.629546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:09.629848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.631253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.637932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.650519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 292/578] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.652097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.697360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc19550-840a-4820-84d1-b54f65f6fe35 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.733028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.733803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.733986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.734226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.747154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:09.751905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:09.752888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 287/579] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:09.753751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.765737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec61440-2cd1-4c7d-9a72-096b2ab95e9d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.801248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.801861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.802138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.802425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.803086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:09.803384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.805279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.813327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 293/580] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:09.813937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.841484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.866825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f9593-9af1-4903-9092-0188b4e2ad5b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.868785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:09.916180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.917181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.917181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.917386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:09.929129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:09.931282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:09.946621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:09.947689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:09.948601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:09.948601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:09.948601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:09.948601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 288/581] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:09.948601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:09.957688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7039f1-8871-42a4-b047-e8d69bd318d6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:09.991722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:09.992868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:09.993147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:09.993416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.000250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.001696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:10.014097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:10.015284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:10.016202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:10.020166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 294/582] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:10.020843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.053595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924abbfd-52e3-4a96-a0eb-1edbb05c4d50 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.088778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.089377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.089656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.089924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.090738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.091159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.092530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.103031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.105687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 289/583] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.106341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.121803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a63df-3861-47ab-8346-014427a1b5f8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.125549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.157012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.157642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.157914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.158182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.169107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.170575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:10.185236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:10.186243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:10.187208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:10.187208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:10.187208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:10.187208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 295/584] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:10.187208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.216376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27f60-404b-48c2-a3bd-bca21706699d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.253538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.254151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.254449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.254726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.255369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.255675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.257062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.269169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 290/585] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.269983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.296801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2b016-e78b-4688-86f2-2e3aae0aa0c9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.331615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.332531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.332987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.333443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.334204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.334621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.336149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.337762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.354019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 296/586] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.354019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.357331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.384214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28591132-ed93-45e0-a9c0-1c50167af13a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.421530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.422161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.422467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.422738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.433327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.434816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:10.449189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:10.450347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:10.451634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:10.451634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:10.451634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:10.451634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 291/587] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:10.455439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.460163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f8200-7de2-48de-8882-e22a95b928f6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.499697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.500479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.500762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.501037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.501681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.501985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.503402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.515531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 297/588] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.516199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.562957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95791cb-3d6a-4c62-ac85-23e2e8436aa2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.573533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.591022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.598950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.599581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.599864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.600133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.607618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.609082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.620863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f184e47-cfe3-42de-b9b3-096bddaaa9ce None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:10.621130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:10.622005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:10.623038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 292/589] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:10.623535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.664596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.665776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.666226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.666671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.682921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=a81fbda6880a41be8922a2f8978e7f21, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.684555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.730793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18277cd9-d73c-486a-99dd-9a9e6cfc78ea None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.764954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.765564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.765848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.766148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.766822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.771619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.771619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.774687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 298/590] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:10 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:10.775702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.783879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 293/591] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.784830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.845220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.908013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321dcbe-d10b-4bde-9aea-d5e05a22d2fa None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.939478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b24fa2-5a9a-4157-8ec5-63cb7805916b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:10.941870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:10.953257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.959604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:10.974426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 294/592] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:10.975420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:10.984026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:10.985149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:10.987985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:10.987985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:10.996275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:10.997744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:11.015552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:11.016529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:11.017481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 299/593] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:11.018092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.089435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe07db-3cf0-4eed-abca-5a32fbaa0f42 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.098940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.124384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.125090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.125346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.125613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.126935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54dc5d-ecad-4e1d-bb4b-ff86e802832f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.134889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.136458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:11.152241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:11.153249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:11.154214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:11.154801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 295/594] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:11.155351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.169210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.169834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.170119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.170392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/fc91470011c746488ea56ca5d9e7a8e4/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.186274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=fc91470011c746488ea56ca5d9e7a8e4, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.187817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.252548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 300/595] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:11 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/fc91470011c746488ea56ca5d9e7a8e4/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:11.253675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.265935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac15478-bf30-4162-a159-91a1b94d240f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.280293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.305834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.306481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.306760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.307796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.307796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:11.308078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.310010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58310b08-fc1e-4934-9ee3-289757e3bb31 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.338170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 296/596] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:11.339551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f9ea0-1827-470c-9c61-8e2817e31dc3 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.339832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f9ea0-1827-470c-9c61-8e2817e31dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.340102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f9ea0-1827-470c-9c61-8e2817e31dc3 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.341365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 297/597] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:11.348241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f9ea0-1827-470c-9c61-8e2817e31dc3 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.388356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.395636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73310127-3d4f-47c8-8574-812241a0c66d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.441349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.441907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.442203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.442471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.443208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:11.443512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.444970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fde434-c048-4be9-a44c-59ea33cf0873 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.465155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 301/598] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:11.466740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.467245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.467731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.486223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:11.487795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:11.495666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f9ea0-1827-470c-9c61-8e2817e31dc3 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.544349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.545017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.545303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.545569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.562495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.563988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 302/599] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:11.564121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.571130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.573894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:11.581088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:11.582353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:11.583334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:11.583334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:11.583334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:11.585286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 298/600] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:11.590947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.691195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.715086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4f614-8b30-42cd-b8f0-5e541e80ae15 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.754540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.755464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.755996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.756289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.756993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:11.757305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.758920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.772657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 299/601] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:11.778969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.831220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543cfea-384b-432d-919b-a0100b76f1c5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.894520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.895170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.895449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.895722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.907979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.909443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.918618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961515d3-92e6-4a71-ab6a-bb4925aad61e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:11.925176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:11.926329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:11.927453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:11.927453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:11.927453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:11.927453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 303/602] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:11.927453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:11.964541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:11.965226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:11.965446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:11.965716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:11.974622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:11.977377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:11.979751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.985006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:11.993770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:11.994895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:11.996021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 300/603] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:12.000352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.036313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d252865-2191-4a4b-9d0d-993ae512a95b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.072947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.073576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.073898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.074176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.084775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:12.090545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:12.091581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:12.092637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 304/604] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:12.093399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.134736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9ae48-fcd5-4553-94fb-54c97615373d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.179296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.180427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.180700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.181219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.182323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:12.182657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.184560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.201195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 301/605] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:12.201377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f47b-40a2-4dfc-806d-59185e5026ae None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.202022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.216413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.239618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.240475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.240928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.241347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.242788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:12.243395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.245029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.264777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 305/606] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:12.265924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.270111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.314434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba262b-ceba-4cfa-8a69-98d40d2eda2c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.349325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.350245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.350698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.351115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.352454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:12.352896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.354450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.368171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 302/607] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:12.369227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.372834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b63a-c9e6-470d-b928-bf0d3a7e078a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.409591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.410224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.410534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.410809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.418571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.420091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:12.432559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:12.433667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 306/608] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:12.434806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.472086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.478813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01572b5-9feb-4cd3-9f70-6983a505c246 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.501469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.516216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.517127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.517660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.518079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/b6cb1f27ea5c49c9a805a9cdd678bef6/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.533216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=b6cb1f27ea5c49c9a805a9cdd678bef6, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.534996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.545780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd309d-72ed-4c26-a2f8-e5f3d8dc30f0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.581203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.581835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.582110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.582377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.593232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.594696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:12.609670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:12.610702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:12.611650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:12.611650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:12.611650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:12.611650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 307/609] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:12.611958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.654697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 303/610] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:12 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/b6cb1f27ea5c49c9a805a9cdd678bef6/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:12.666465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.786109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18272f-a7d0-4d13-bb29-2248757dcdfa None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.795798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.813687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebc453-f237-46eb-bb74-1701a893be3a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.826530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.827603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.827999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.828368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.829323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:12.829619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.831146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec72f54-b379-4e07-9282-745c78ab064e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.845753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 304/611] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:12.847317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44728-4d04-40e1-b1e3-8c16dedb0d74 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.847706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44728-4d04-40e1-b1e3-8c16dedb0d74 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.848078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e44728-4d04-40e1-b1e3-8c16dedb0d74 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.849697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 305/612] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:12.852495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e44728-4d04-40e1-b1e3-8c16dedb0d74 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:12.862004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:12.862697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.862973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.863361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.864060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:12.864361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:12.865859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db94be-60ee-4c94-b73f-b76a54481c7e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:12.879131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 308/613] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:12.881150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:12.881506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:12.882001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:12.893301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:12.902874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:12.904160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:12.974287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e44728-4d04-40e1-b1e3-8c16dedb0d74 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:12.978211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 309/614] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:12.979158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.009819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.010476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.010763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.011038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.019188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.020680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:13.033180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:13.034354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:13.035375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:13.039710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 306/615] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:13.040534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.064996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:13.103685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb2aa0-de24-4c62-96c0-2141a9c21c9f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.160350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.161052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.161330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.161603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/1d00e1ef6c1a4f63a9a4514aac586c67/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.177354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=1d00e1ef6c1a4f63a9a4514aac586c67, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.178848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.206863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:13.213552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a108fd8-9ce9-4d40-8ae5-a7e115df7afa None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.255664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 310/616] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:12 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/1d00e1ef6c1a4f63a9a4514aac586c67/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:13.256689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.257367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.258223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.263795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.263795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:13.263795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.263795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcab53-a0c1-4971-8d5b-84bcbf5b5d04 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.292465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 307/617] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:13.297793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.400816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db6fb-62a4-46e9-9651-9e1e8ca4b6ce None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.401148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db6fb-62a4-46e9-9651-9e1e8ca4b6ce None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.401432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240db6fb-62a4-46e9-9651-9e1e8ca4b6ce None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.402810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 308/618] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:13.408705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.408988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.409267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.433915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:13.435328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:13.439423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb1da1-c500-4fe8-808b-6fcf6ed3dfe7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.468921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:13.478314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.478975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.479251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.481400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.493164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.494678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:13.511531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:13.516360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:13.517355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:13.517355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:13.517355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:13.517355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 311/619] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:13.517355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.522683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 309/620] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:13.528447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.666400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e435c09-a588-48ec-a989-3cc30c54bbc6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.704131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.705723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.706004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.706278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.720724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:13.722812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:13.733234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:13.739342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:13.740194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 312/621] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:13.740914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.772595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bb2b4-17fb-42c9-8578-cb9f87f67f6b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.789804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:13.833928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.834604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.834864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.835136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.835981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:13.836292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.837688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.850449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 310/622] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:13.856385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.889864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5f64-d6b9-47cc-ad47-88555248c879 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:13.939738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:13.940350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:13.940626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:13.940898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:13.941570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:13.941870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:13.943943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:13.958494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 313/623] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:13.959684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:13.982134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16a64f-2fde-4883-8d68-623d5eddd0da None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.028990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.029758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.030166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.030484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.043099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.045071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.047294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:14.061722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:14.065379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 311/624] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:14.066314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.070151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.074097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc77b-b009-4c8c-acab-451bed74d3f2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.119723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.120664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.120795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.121042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.138768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:14.145303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:14.146242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:14.147149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:14.150479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 314/625] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:14.152217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.184359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88428f8a-0c7c-434d-af8f-c3e3a00c9892 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.221478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.222612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.223184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.223649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.236434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.238244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:14.253584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:14.254620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:14.255715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:14.255715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:14.255715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:14.257516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 312/626] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:14.258563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.279981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570b489-1b46-4c4c-8bfb-7ea885bd5513 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.319405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.334292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.334929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.335202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.335543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.336684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:14.336986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.338438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.357813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 315/627] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:14.359506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.388873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.389172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b855b87-9eaa-4c66-a32e-c2d3a930440c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.427299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.427618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.427838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.428171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.428884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:14.429197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.430764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.440851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 313/628] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:14.441957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.491490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416caa-f13b-4fcf-b732-8f96a78155f5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.529104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.530046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.530568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.531129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.532076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:14.532571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.534905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.547694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c271b-89db-4dc7-aaa7-fc210cf0bf76 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.548528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 316/629] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:14.549653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.561821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.584391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.585106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.585400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.586321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/01a4e7f4d1e54b17938c3514428a34b9/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.601528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=01a4e7f4d1e54b17938c3514428a34b9, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.603100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.642609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.660525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 314/630] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:14 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/01a4e7f4d1e54b17938c3514428a34b9/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:14.662102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.663222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f654e-24c5-446c-b091-bde2facb31c0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.704440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.705286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.705722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.706089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.719783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.721602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:14.738991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:14.740229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:14.741206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:14.741206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:14.741206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:14.741911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 317/631] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:14.747381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.806743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7179a-d87b-440d-a619-bae7960f691a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.858109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.859641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.860141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.860609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.869434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.871225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-6a875201-cdaf-468f-b5bf-0eaab929bc1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:14.886234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:14.887373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:14.888367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 315/632] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:14.888986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa0e6-e2b0-4180-a4ba-3172f3fa9be7 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.890024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa0e6-e2b0-4180-a4ba-3172f3fa9be7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.890024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffa0e6-e2b0-4180-a4ba-3172f3fa9be7 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.891439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 316/633] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:14.895259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ca6f5-5a52-434b-9404-98361d67179c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:14.897617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa0e6-e2b0-4180-a4ba-3172f3fa9be7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:14.927978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.934172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:14.935131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.935782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.936258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.937331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:14.937720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:14.938607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:14.940178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe9472-0f1e-47f8-ab56-c68cd34746a6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:14.957722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 318/634] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:14.959405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:14.959878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:14.960399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:14.983278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:14.984939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:15.031349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffa0e6-e2b0-4180-a4ba-3172f3fa9be7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.062809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 319/635] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:15.063801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.071396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.072441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.072929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.073416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.074499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:15.074987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.076792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.101434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 317/636] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:15.103049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.174350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242af652-0db3-44db-bbad-7ca1c26c8d7d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.192309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.205439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.214249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26231-1578-43de-accf-dd8b55305a90 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.219726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.220365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.220646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.220920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] PATH_INFO: `/v3/users/glance` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.229891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.231804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:15.247601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:15.249002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:15.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 320/637] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:15.253034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.276329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.277388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.277869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.278440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.371985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca8d4c-9958-4540-a908-4d73942522da None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.409926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.410679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.411400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.411686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.411964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.412703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:15.412967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.413070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.414975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.438758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 321/638] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:15.439952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.485897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.499883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.519299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 318/639] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:15.519970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.551416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bf2a3-1ae9-432f-85e3-00eb81cd7774 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.588938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.589561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.589839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.590109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.600362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7381b574-0161-448c-90c6-821f599a7cb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:15.605809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:15.606792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:15.607660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 322/640] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:15.608364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ec33-9c58-407a-a855-88582aa05150 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.608433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ec33-9c58-407a-a855-88582aa05150 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.609233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531ec33-9c58-407a-a855-88582aa05150 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.613861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 323/641] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:15.613861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531ec33-9c58-407a-a855-88582aa05150 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.627177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73fe6e-1a8e-4200-bc5f-2b9eb47c9941 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.667882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.668732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.669009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.669731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.720368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.725334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531ec33-9c58-407a-a855-88582aa05150 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.730919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.750299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.751795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba202d8c-d803-49fe-b5d8-f787cfc6f0ff None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.762104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.763982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.763982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.763982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.774921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.776362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:15.792404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:15.793628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:15.794831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:15.794831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:15.794831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:15.794831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 324/642] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:15.794831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.872160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 319/643] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:15.872160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.872160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.872160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.888477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:15.890002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:15.919933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35e283-8ccb-4b28-8197-e05feaf5b747 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:15.955558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:15.956167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:15.956554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:15.956824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:15.957522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:15.957827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:15.959264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:15.966919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 320/644] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:15.966919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.970824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 325/645] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:15.971590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:15.979998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:15.992724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.088036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfab69-703a-44af-8d1b-bf7b795cf039 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.104263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14f5b8-109a-4639-b6b2-cd4d1a9ffa55 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.123557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.124165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.124443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.124714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.135186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.136698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.149411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.150282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.150811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.151303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.152164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:16.152416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:16.153348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:16.154970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:16.154970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:16.154970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:16.154970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 326/646] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:16.154970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.155733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.157649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.178793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 321/647] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:16.181062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.236028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.262024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.277804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100218-9316-474c-a2fb-433ac099f272 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.295267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881e5ef-e3ca-446b-b262-04be3c56aaec None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.317321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.317961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.318247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.318555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.329220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.330705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.331546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.332250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.332538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.332810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.333564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:16.333868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.335264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:16.344956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:16.345936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:16.347311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:16.347311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:16.347311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:16.347311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 327/648] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:16.347311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.357672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 322/649] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:16.357672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.463372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff059ef-24b5-4db7-a045-a118df747581 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.464125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd5ae9-dd31-48c9-82ea-f31a1932d99c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.483221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.496634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.502591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.503230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.503509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.503674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.503787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.504286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.504566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.504839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.514131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.515777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:16.516154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:16.520700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:16.527267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 323/650] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:16.528207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:16.535689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:16.536599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:16.537505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 328/651] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:16.539384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.650042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c1343-8528-4365-b24a-daab0282e905 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.651244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d937-e5e7-4799-bd3e-92e468d3f24f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.687955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.688200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.688565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.688672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.688845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.688955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.689106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.689198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.689749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:16.690054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.691563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.696146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.697573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.703059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 324/652] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:16.707312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:16.710235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:16.711768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:16.712716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:16.717765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 329/653] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:16.718535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.761110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.764759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.807577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5b2c5-c3d6-48c0-8042-e57b5b7778df None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.825931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106dc863-fdcf-40be-a646-f2987d754987 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.842811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.843425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.843702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.843968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.844706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:16.845005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.846389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.854774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 325/654] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:16.855560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.869687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.870287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.870597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.870865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.871654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:16.871952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.873365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:16.884691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 330/655] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:16.885436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:16.951899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ecf92-a2e1-46db-8aaa-d163697071f9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.964224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.982304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:16.984505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:16.985130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:16.985408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:16.985675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:16.986189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839f468-adc1-4a5f-aefd-ca681dafe8d5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:16.993070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:16.994552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1af2021e-fad7-4761-816f-b83de65a4141 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:17.005606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:17.006612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 326/656] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a1e-4e26-454e-97b8-16feeb4b4b95 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.017846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a1e-4e26-454e-97b8-16feeb4b4b95 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.018636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a1e-4e26-454e-97b8-16feeb4b4b95 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.018636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 327/657] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:17.018636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431a1e-4e26-454e-97b8-16feeb4b4b95 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.022435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.023601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.023872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.024138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.024820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:17.025115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.026760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.038463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 331/658] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:17.039275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.111833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431a1e-4e26-454e-97b8-16feeb4b4b95 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.135345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b6f93-8d01-4aba-af3b-91edb4f0c61c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.144436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.145063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.145344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.145614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.156968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.158445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.169487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.170078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.170357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.170661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.171380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:17.171689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:17.171998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:17.173072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:17.173984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:17.173984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:17.173984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:17.174167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fb194-5554-4d36-87af-9104c5bef113 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.174244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 328/659] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:17.174244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.182468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 332/660] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:17.183886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.184163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.184432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.203644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:17.204888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:17.220103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.226566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.294793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 333/661] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:17.296290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.311961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd62aa-2a33-4eec-8f60-e3c0cf8f47cc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.363096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.364487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.365202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.365689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.367617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:17.368166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.370052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.389887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 329/662] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:17.391009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.428476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2fe1a-e863-414e-b2ab-abcf8750cd10 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.468716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.470225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.470225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.470225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.470767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=fc91470011c746488ea56ca5d9e7a8e4) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:17.471075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.472621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.480013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 334/663] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/role_assignments?user.id=fc91470011c746488ea56ca5d9e7a8e4&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:17.481108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.495049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.509953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.526942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bed383-4694-4a00-b36c-e03c973867d9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.599482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.601967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.602464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.602908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.603746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:17.604388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.606696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.624523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 330/664] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:17.626138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.638709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511ef35-5046-4c23-a187-453bd451d1c1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.689924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.690491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.690770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.691096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.757535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2930b06-ad80-4f0f-b5d7-e46ebb90fcd6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.783684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.785164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:17.801117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.802307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.803197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.803819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.811577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.821570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:17.827969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:17.829724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:17.830697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:17.834456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 331/665] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:17.835295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.838811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:17.920239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 335/666] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:17.921169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:17.950568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd1800-78d9-44ab-81cd-94e7ef9d400a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:17.987155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:17.988592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:17.989078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:17.989550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:17.997706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:17.999440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:18.011051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:18.012094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 05:11:18.013072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:18.016060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 332/667] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:18.017167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.049694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26de7f-2fbf-4a0c-9b05-109ff9ff1226 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.067585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.087471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.088423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.089036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.089513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.090381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:18.090909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.092504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.095966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.102536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 336/668] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:18.103622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.152174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf49ebc-c087-4be4-a174-4ab9220db868 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.196116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.196759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.197042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.197315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.210162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.211934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.217446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52ce90-6109-4148-bd79-9dcbc5750bb2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:18.227791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:18.228710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:18.228710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:18.228710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:18.230932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 333/669] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:18.232053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.262282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.263168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.263450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.263731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.265139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:18.265453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.267492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.291467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 337/670] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:18.292326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.335849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.354889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966f00b-6f7f-4259-9512-c71dfe62efec None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.369562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.395917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.396604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.396894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.397183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.409263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.410684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.412657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac701db-fa77-4bd7-bb73-651f293bacf2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:18.425432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:18.426450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:18.427373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:18.427373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:18.427373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:18.430772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 334/671] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:18.431646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.453378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.454457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.454952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.455618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.456537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:18.457096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.458758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.472523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 338/672] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:18.473783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.549185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b597c93-c554-47d3-8034-7656be42b216 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.574538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.581040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d71fa4-66d5-4aa7-aa2d-1e3280a62bc9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.587177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.587177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.587177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.587177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.625321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.626027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.626776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.627636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.628919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.630275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:18.630781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.635401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.653171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 339/673] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:18.654884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.666060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.667707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.770657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9a41d-ba4f-489c-8ae1-0d527a78fe4c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.775903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 335/674] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:18.776890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.808457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.809333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.809877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.810368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.817676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.819430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:18.831635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:18.832563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:18.833485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:18.840860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 340/675] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:18.841953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.846903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.862650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:18.884537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90df78-7e18-45bd-aebc-31c7d9af31e2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.921477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.922118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.922400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.922695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:18.930169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:18.931841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:18.943851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:18.944898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 336/676] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:18.946487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:18.955715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1a460-f91e-4ade-a40d-09b6d3a07a7b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:18.992682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:18.993706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:18.994220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:18.994732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.006931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.008789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:19.024818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:19.025807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:19.026771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:19.026771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:19.026771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:19.028887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 341/677] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:19.029956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.055306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e58f1-bc6d-4a29-8e57-fcf1162a0126 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.082566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.092283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.096208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.102830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.114009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 337/678] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:19.114009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.163668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdcc1e-2427-40f3-8923-64d796c25f42 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.206720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.207339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.207628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.207906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.208651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:19.208959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.211753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.226779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d921-fafa-4385-97f7-3418d3e28645 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.238667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 342/679] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:19.241675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.275618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.276653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.276945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.277218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.278017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:19.278456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.280222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.298440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 338/680] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:19.298440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.355663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.372963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fff8a-dc44-4d6d-a0ba-08d094739edb None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.401013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.411022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.411638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.416040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.416040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.416218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2a860-4f73-481b-b086-2461ded392be None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.416298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=b6cb1f27ea5c49c9a805a9cdd678bef6) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:19.416298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.416298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.430707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 343/681] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/role_assignments?user.id=b6cb1f27ea5c49c9a805a9cdd678bef6&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:19.430707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.471053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.471722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.472549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.472835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.553743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823265-85a0-46c4-a392-b2e928dedc6f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.577983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.579657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.594303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.594974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.596019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.596287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.605377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.606864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1a80d878-0f87-4aa6-b344-b04de736347e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:19.621901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:19.622886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:19.623800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 344/682] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:19.624863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e5961-652c-43a4-bfe7-72d7d87a1f47 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.624863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e5961-652c-43a4-bfe7-72d7d87a1f47 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.625139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e5961-652c-43a4-bfe7-72d7d87a1f47 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.626652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 345/683] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:19.627522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42204d-954b-4846-9384-1b5be8a207f5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.627863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42204d-954b-4846-9384-1b5be8a207f5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.628140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42204d-954b-4846-9384-1b5be8a207f5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.629614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 346/684] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:19.631910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7200-3626-4b6a-848f-5366c2e9a715 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.632251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7200-3626-4b6a-848f-5366c2e9a715 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.633010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d7200-3626-4b6a-848f-5366c2e9a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.639411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.655628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-712d7200-3626-4b6a-848f-5366c2e9a715 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:19.657434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-712d7200-3626-4b6a-848f-5366c2e9a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:19.706368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 339/685] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:19.706368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.711135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:19.735152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 347/686] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:19.736558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.736848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.737121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.771246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:19.772444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:19.814684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4089e3-03a7-4e6c-92ac-e6c571e0bb4d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:19.854284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:19.855104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:19.855329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:19.855603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:19.856295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:19.856600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:19.858285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:19.863475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 348/687] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:19.864615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.875884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 340/688] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:19.878767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:19.894166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.025546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6bd44-2868-45b9-ade5-416b270bbed3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.031333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29793cf7-7a3a-45f1-85d0-1ffba321968b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.052089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.062460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.063134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.063422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.063670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.070769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.070769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.070769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.070769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.071473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=1d00e1ef6c1a4f63a9a4514aac586c67) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:20.071473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.071473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.074711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.076156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.076763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 341/689] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/role_assignments?user.id=1d00e1ef6c1a4f63a9a4514aac586c67&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:20.077624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:20.092447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:20.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:20.094833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:20.094833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.094833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:20.096902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 349/690] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:20.103020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.196417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.232058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67824f4a-f070-43c2-ae48-f65ecdedccc1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.237916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a816-becf-430d-9ba8-9cf508c03024 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.275250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.276540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.277266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.277547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.284218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.284831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.285108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.285379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.286023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:20.286326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.287778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.291962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:20.302155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:20.303208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:20.304604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:20.305297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 350/691] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:20.305297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.307287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 342/692] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:20.311139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.336997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.441130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86f258-a7aa-41c8-b46c-074896a50163 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.452038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806eeda9-b986-4be0-bbef-e41c20a671cc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.483166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.484833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.485321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.486053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.487579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.494822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.495458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.495738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.496008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.501319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.503349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.511159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.511330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:20.517918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:20.518878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:20.519796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:20.524035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 343/693] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:20.524306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:20.525216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:20.526121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:20.526121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.526121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:20.526293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.526366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 351/694] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:20.527123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.622985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.664461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80c23-242b-4bf5-866b-e7e84b4fe3bd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.665543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff219fc-858d-4f25-a672-5809a6bc1bf8 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.701936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.702538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.702799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.703068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.703728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:20.704029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.705453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.712219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.713120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.713554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.713990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.714838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 344/695] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:20.715000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:20.715496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.715978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.717279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.736350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 352/696] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:20.737408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.777285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.831647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194a61a-b015-45b7-be88-0fe07c7159bf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.864248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd98ed6-ba38-4de4-8c3c-4214f3a825ee None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:20.878817 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.880083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.880650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.881200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.882002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:20.882718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.884649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.902851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 345/697] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:20.903060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:20.903837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:20.904061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:20.904441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:20.904854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:20.906955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:20.917017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:20.918832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:20.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:20.934842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:20.935966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:20.935966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:20.935966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:20.938310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 353/698] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:20.939324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:21.008000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ae3c0-f9f0-4b26-b331-c2571d4dc603 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.023428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:21.043803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba6581-b4a0-483b-aeb1-a995d73c9749 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.044865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.045462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.045741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.046010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.046757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=01a4e7f4d1e54b17938c3514428a34b9) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:21.047057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.048501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:21.055446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 346/699] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:20 2022] GET /identity/v3/role_assignments?user.id=01a4e7f4d1e54b17938c3514428a34b9&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:21.056279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:21.082693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.083705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.084204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.084674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.086062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.087773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:21.176624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f3ff3-b716-4c92-9c6f-b4e7512e68c7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.209329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 354/700] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:11:20 2022] POST /identity/v3/users => generated 254 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:21.221250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.221889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.222165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.222454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.223208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:21.223518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.224990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:21.245473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 347/701] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:21.251567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:21.303475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:21.373405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c203c9-d56b-407c-a9b7-385d5f6aabf5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.412253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.412885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.413165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.413469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.421886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.423424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d112d712-ff74-4297-8926-e2ea828a0ec2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:21.434900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:21.435878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:21.436874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:21.438488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 355/702] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:21.441983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:21.486328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:21.556861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a75c22-3e08-4408-9b4a-339b0a8bdb7d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.593638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.594254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.594658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.594917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.595683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:21.596000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.597449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:21.607014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afebc5-c9e2-44c9-8b8e-c828940a6f5a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.607623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afebc5-c9e2-44c9-8b8e-c828940a6f5a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.608280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afebc5-c9e2-44c9-8b8e-c828940a6f5a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.610378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 356/703] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:21.619970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.620260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.620607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.621907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 348/704] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:21.628296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:21.660507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:21.662126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:21.724587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:21.731152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba175b-88e0-4c0b-903c-b9c52d9da1da None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:21.779229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 357/705] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:21.780802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:21.781428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:21.781705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:21.781979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:21.884176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:21.885660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.016420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.024740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 349/706] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:22.030266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.091353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.170898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.183212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe436ec9-46bf-43b8-83f1-f6a4296a7370 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.254301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.254973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.255219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.255471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.270284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.272468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8a833be9-a9dc-47a0-8a7f-8e545c06be8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:22.291427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:22.292393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:22.293298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:22.293298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:22.293298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:22.293298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 358/707] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:22.298430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81180-aa6b-4b07-be6a-d73bb20de813 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.298430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81180-aa6b-4b07-be6a-d73bb20de813 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.298430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a81180-aa6b-4b07-be6a-d73bb20de813 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.298430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 359/708] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:22.298430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a81180-aa6b-4b07-be6a-d73bb20de813 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.321233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a2dc5-d1c2-44a4-b647-3a9f768be1ee None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.357956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.358619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.358897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.359167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.359744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.361164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-406e1031-558f-4c62-a99b-a331045706bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:22.371868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:22.372935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:22.373813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 350/709] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:22.374670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.374754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.375017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.393424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:22.394655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:22.412249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.434989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a81180-aa6b-4b07-be6a-d73bb20de813 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.455243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.474439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 351/710] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:22.474439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.495532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.496136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.496410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.496675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.497315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:22.497612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.498997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.520560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 360/711] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:22.523667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.666682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5bc15-06de-472d-8317-5d2d9fa8ae49 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.670566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5c26b-34f8-40c8-88eb-059bf309d15f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.708076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.708916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.709381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.709723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.710485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:22.710857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.713675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.718131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 352/712] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:22.730572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.747213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.748230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.750122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.750122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.762259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.763782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:22.778243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:22.779413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 361/713] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:22.780564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.785164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.793940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:22.843075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0661241-6e69-44fa-984f-b5dff9bfb327 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.882681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.883289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.883565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.883833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.884459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:22.884759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.886332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.890078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83508e0-e40e-4ba1-87e1-015c5aa43253 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:22.892593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 353/714] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:22.893855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:22.933621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:22.934577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:22.935067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:22.935504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] PATH_INFO: `/v3/services/network` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:22.936365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:22.937969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:22.950767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:22.951953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:22.952868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:22.955391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 362/715] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:22.961466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:23.006444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ee38c-2261-45b2-b30a-57b54f255ee5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:23.024420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:23.049880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:23.051147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.052108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.052660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.055117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:23.056938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:23.058546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:23.059853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:23.073866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 354/716] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:23.079938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:23.104738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b76440-94e5-4928-b3e9-15a6d0f4e4bc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:23.154844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:23.155605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.155997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.156362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.157786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] RBAC: Adding query filter params (name=network) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:23.159002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:23.160584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:23.167559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 363/717] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:23.173599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:23.208649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1aa19-f1d9-46b0-bbd2-7a15faa281ea None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:23.249471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:23.250286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.250733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.251100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.252079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:23.252645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:23.254871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95368746-1bbb-4397-90cf-9602d93f5f5a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:23.262626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 355/718] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:23 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:23.296785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb3bea-abbf-443e-a38a-00e8a2ff8e32 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:23.332576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:23.343328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:23.344265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.344705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.345127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.345969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] RBAC: Adding query filter params (type=network) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:23.346464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:23.348062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3454cc-5aca-4222-bac3-1fa52a53a123 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:23.354781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 364/719] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:23.550906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02972d-5a92-420c-b67a-dc66f245d48a None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.551420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02972d-5a92-420c-b67a-dc66f245d48a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.551420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02972d-5a92-420c-b67a-dc66f245d48a None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.553068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 356/720] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:23.558870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d71f8-c788-4b75-8b00-001a40ef7f5e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.559163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d71f8-c788-4b75-8b00-001a40ef7f5e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.559521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d71f8-c788-4b75-8b00-001a40ef7f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.581224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e97d71f8-c788-4b75-8b00-001a40ef7f5e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:23.582755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e97d71f8-c788-4b75-8b00-001a40ef7f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:23.680080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 365/721] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:23.777058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e348-706b-4e3e-9043-4254b1ff7693 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.777406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e348-706b-4e3e-9043-4254b1ff7693 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.777600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6e348-706b-4e3e-9043-4254b1ff7693 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.779238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 357/722] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:23.784208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:23.784502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:23.784780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:23.807424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:23.808627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:23.899874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 366/723] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:24.103827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e348-706b-4e3e-9043-4254b1ff7693 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.164978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:24.238234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6e348-706b-4e3e-9043-4254b1ff7693 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:24.272338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:24.273477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:24.273755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:24.274022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:24.279551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.296623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:24.304413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:24.305512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:24.306377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:24.307051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 358/724] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:24.311341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.400678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:24.449042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:24.453808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e49be-299c-4da4-bbf3-a75129040538 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:24.483767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ae3f6-445f-46b8-a90d-b3cdca496889 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:24.514935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:24.515552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:24.515831 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:24.516104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:24.516778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:24.517079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:24.519497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:24.531227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 359/725] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:24.537058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.573452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:24.574099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:24.574378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:24.574666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:24.599445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:24.604965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:24.606026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:24.606907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 367/726] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:24.620238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.713364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b150c41-bd61-49d2-88da-9e808ad7d407 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:24.750734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:24.751451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:24.751796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:24.752152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:24.770356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:24.771837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:24.796278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:24.797309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:24.798275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:24.798275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:24.798275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:24.798483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 360/727] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:24.805262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:24.828071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a62ffe-0676-40af-8b6f-60f1d4c00c3d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:24.839867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:24.854451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:24.880376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:24.881232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:24.881651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:24.882142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:24.882995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:24.883460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:24.885096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:24.894071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 368/728] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:24.900296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.020296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842affa-f45d-44cb-beff-f73dba9c40b2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.081624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.082867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.084286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.084731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.086374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:25.086841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.088971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.097253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68379abb-e68a-45a0-bc82-c1293d966b09 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.111655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 361/729] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:25.116888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.167151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.167801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.168081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.168352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.195873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.197344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.204934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:25.231846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:25.233126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:25.234240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:25.234240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:25.234240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:25.234240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 369/730] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:25.240106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3aebb-0580-4baf-ac03-c2be3d943ad6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.241488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.281535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.282434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.282982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.283399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.292115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.293799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:25.307415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:25.308576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:25.309715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:25.312177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 362/731] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:25.317972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.340253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.435359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e1f3c-a25e-4c31-be0c-229793a92c0f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.472238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.473128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.473547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.473961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.474794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:25.475317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.476903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.489396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 370/732] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:25.495625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.532459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6a863-30fa-4142-b5d6-645118bf8b0b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.576588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.577198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.577481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.577752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.578503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:25.578813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.580257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b44d1-3898-4c4b-bcaa-d681a21c42a9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.582141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.592652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 363/733] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:25.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f7e0-b792-43aa-9858-b55ce211aa31 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f7e0-b792-43aa-9858-b55ce211aa31 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f7e0-b792-43aa-9858-b55ce211aa31 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 364/734] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:25.603672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c9f5-2d7e-4bfd-a3b0-f8a7930ea2a8 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.603929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c9f5-2d7e-4bfd-a3b0-f8a7930ea2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.605311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6c9f5-2d7e-4bfd-a3b0-f8a7930ea2a8 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.606272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 365/735] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:25.607010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6c9f5-2d7e-4bfd-a3b0-f8a7930ea2a8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.610452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f11e-cab3-4ef4-97a9-bd94f61d4527 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.646492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.647163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.647470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.647743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.655147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.656604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-594ac63f-d3a9-4288-94ef-214eee94475b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:25.672279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:25.673287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 371/736] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae61d2-c049-4177-be5a-5cf05f1541cd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.679314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae61d2-c049-4177-be5a-5cf05f1541cd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.680070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae61d2-c049-4177-be5a-5cf05f1541cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.697374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-66ae61d2-c049-4177-be5a-5cf05f1541cd None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:25.698614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-66ae61d2-c049-4177-be5a-5cf05f1541cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:25.727216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6c9f5-2d7e-4bfd-a3b0-f8a7930ea2a8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:25.761006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:25.761606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.761890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.762158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.766303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 372/737] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:25.767630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.767915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.768185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.792098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:25.793209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:25.832941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.842483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:25.844020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec915bd5-4009-4865-81ee-1a71aa108f93 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:25.882435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 373/738] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:25.882435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.883444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:25.959468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 366/739] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:25.960985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0083-9fea-4e6a-bed3-9f558a342b5e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:25.961361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0083-9fea-4e6a-bed3-9f558a342b5e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:25.961706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0083-9fea-4e6a-bed3-9f558a342b5e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:25.963329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 367/740] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:25.964894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0083-9fea-4e6a-bed3-9f558a342b5e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:25.978578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d088c5-13fa-4634-8cac-5c6a3b5b1013 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.015843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.016563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.016844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.017117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.017971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:26.018277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.019761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9ef34-fe11-4fb3-bb6f-99f32dcde8d7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.038617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 374/741] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:26.039919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.040201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.040534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.059966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:26.061078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:26.064875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.073666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0083-9fea-4e6a-bed3-9f558a342b5e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.108225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.109181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.109483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.109757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.121232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.122723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.133591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:26.138149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:26.139621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:26.140715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:26.140715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:26.140715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:26.140715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 368/742] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:26.142315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 375/743] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 05:11:26.143306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.144913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.248734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8a82f-6e28-413f-a13f-1be805828365 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.260946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe8343-4d2d-4159-b7fb-f2ece9b0a6ff None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.286958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.288087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.288986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.289273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.295162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.295781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.296057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.296323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.306622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:26.311542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:26.312467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:26.313330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 369/744] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:26.314041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.318561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.362631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.366678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.368066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.420954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2f1be-495b-4444-82ce-6349f76efdca None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.453054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.453656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.453931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.454201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.454889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:26.455189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.456561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.467379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 376/745] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 05:11:26.468199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.472541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 370/746] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:26.473395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.533204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.563810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.567767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77cea-f46b-4c9b-ac39-0f1ad5547994 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.570255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab3941-7991-451a-ab98-cd896aacf742 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.602119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.602961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.603485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.603612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.603761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.604147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.604412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.604521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.604685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.605341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:26.605642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.605782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.607041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.615098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 371/747] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:26.615871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.701688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 377/748] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:26 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:26.702630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.714231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426925a1-43e2-488d-8b48-392dd70cfed1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.744337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.762550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.763307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.763592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.764124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.764789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.766227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.821271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f9dca-790a-4878-abb3-92eafd115bac None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:26.838481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 372/749] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:26 2022] POST /identity/v3/services => generated 240 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:26.838481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.885857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:26.886843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:26.887291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:26.887739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:26.902094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:26.904052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:26.921459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:26.922533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:26.923649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:26.923649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:26.923649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:26.923649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 378/750] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:26.923891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:26.937434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:26.979024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780fee1-31f2-4172-94fa-3b8ce053c591 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.018670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.019311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.019590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.019857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.028278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.029896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:27.042002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:27.043056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:27.044091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:27.045755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 373/751] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:27.046734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.047669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12005c5d-b4a0-4d22-b25d-9423267440e6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.086285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.086849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.087071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.087339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.098541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.100061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:27.114990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:27.116737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:27.117908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:27.117908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:27.117908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:27.117908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 379/752] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:27.117908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.142873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.186479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16d937-b3f2-4956-9e26-73ef83adba58 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.189362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.237981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.239120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.239585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.240031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.240669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff12e-f13f-4000-861e-6100e0fa8c29 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.241003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.241972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.244754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.262283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 374/753] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:27.263410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.282494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.283258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.283793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.284064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.284939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.285308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.286980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.300551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 380/754] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:27.301788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.372884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13a51d-6af9-455e-b7b3-0c6c652021fe None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.374802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.410389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.411265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.411698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.412149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.413106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.413563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.415162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.415975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b88ff9-e32e-4809-ae61-ccc4143b4473 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.428226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 375/755] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:27.429644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.544588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b136e-cff7-48b1-b9f5-1548ec48c4a4 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.555272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.574820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.575459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.575737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.576008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.582301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.583105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.583245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.583658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.584079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.585517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=d3b59fdbefe24db892bb40b4b23fe78c) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.585981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.586109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.588007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.594248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 376/756] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/role_assignments?user.id=d3b59fdbefe24db892bb40b4b23fe78c&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:27.595265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:27.597959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:27.599018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:27.599940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 381/757] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:27.605015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.630376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.711206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23742f-7abe-4b3a-98de-1bacc93fe368 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.724473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b39ba-5162-46bf-bd5d-609797e6d3d7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.749290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.750727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.751217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.751695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.761132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.763040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:27.776219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:27.777252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:27.778270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:27.780993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 377/758] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:27.787239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.802990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.804557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.805056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.805530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.806457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.807012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.808668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.829153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 382/759] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:27.835612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:27.901252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.913624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447370f-eddc-4d27-b811-3172be248e80 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:27.918814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:27.952805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:27.953792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:27.954312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:27.954852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:27.955811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:27.956338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:27.958003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:27.976349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 378/760] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:27.987424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:28.011614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0517d-86b5-4a61-b607-09902afc72c3 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:28.076782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:28.077394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.077671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.077938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.078701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=01a4e7f4d1e54b17938c3514428a34b9) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:28.079010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:28.080450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:28.090031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 383/761] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/role_assignments?user.id=01a4e7f4d1e54b17938c3514428a34b9&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:28.177073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f1ab-c02c-4f3e-88d5-074265c84065 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:28.223965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:28.224623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.224902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.225174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.237048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:28.238550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:28.241883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:28.258954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:28.260331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:28.261279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:28.261279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:28.261279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:28.261279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 379/762] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:28.267601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:28.370751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e35f97-2875-475b-98b5-55aa49edf12a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:28.372528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:28.407016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:28.407772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.408151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.408522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.409284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:28.409691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:28.411185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:28.423742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 384/763] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:28.428776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:28.549033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81394-96db-4d04-bd63-4142f3f1d904 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:28.595962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:28.596745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.597129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.597527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.611433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:28.613030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:28.625079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:28.626209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:28.627974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:28.628988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 380/764] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:28.641503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:28.759675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6fb2e-5329-4539-adaf-9c76d386470e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:28.815694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:28.816306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.816583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.816856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.817544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:28.817844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:28.820096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c4ebb-2e8b-4f46-8f88-05a352bd07e2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:28.840368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 385/765] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:28.846291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:28.930211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:28.933602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60174a-9b52-493a-851a-e8b722bc89f4 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.933890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60174a-9b52-493a-851a-e8b722bc89f4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.934165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60174a-9b52-493a-851a-e8b722bc89f4 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.936002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 386/766] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:28.941033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a4f9e-7381-4806-97f1-09d6e95bf0de None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:28.941317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a4f9e-7381-4806-97f1-09d6e95bf0de None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:28.941592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a4f9e-7381-4806-97f1-09d6e95bf0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:28.972661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a13a4f9e-7381-4806-97f1-09d6e95bf0de None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:28.973872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a13a4f9e-7381-4806-97f1-09d6e95bf0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:28.996502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:29.019651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e46a02-baf5-4d9c-9060-07016a996645 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:29.065830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:29.066950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.067465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.067975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.072842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=a81fbda6880a41be8922a2f8978e7f21, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:29.074560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1b5de-0a7b-45fa-a59b-45061ba26dc2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:29.090480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 387/767] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:29.153709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 381/768] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:28 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:29.406456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb43-0112-4f21-b1ab-d5c384301518 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.407079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb43-0112-4f21-b1ab-d5c384301518 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.407566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9bb43-0112-4f21-b1ab-d5c384301518 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.409116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 388/769] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:29.415151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.415700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.416177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.417085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9bb43-0112-4f21-b1ab-d5c384301518 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:29.450383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:29.454697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:29.594990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 382/770] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:29.616069 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9bb43-0112-4f21-b1ab-d5c384301518 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:29.657994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:29.658600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.658856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.659124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.659704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:29.661064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:29.674941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:29.675941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:29.677405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 389/771] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 05:11:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:29.681793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:29.741848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:29.802219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1757b-995e-4573-834e-fbf3d7e889b4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:29.841994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:29.843031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.843319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.843599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.844247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:29.844557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:29.846061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58821e7f-dfbb-43e3-a1b3-3464b83e971d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:29.851680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 383/772] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:29.856986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:29.908229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:29.923369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cef4-2a2e-4ee8-9279-bc8d0b1b521d None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.923984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cef4-2a2e-4ee8-9279-bc8d0b1b521d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.924464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845cef4-2a2e-4ee8-9279-bc8d0b1b521d None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.927293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 384/773] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:29.932178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:29.932712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:29.933166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:29.977811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:29.980240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:30.029713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999f2cb-77cb-40e3-981b-18cd4847edb5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.064064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 385/774] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:30.064064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.082610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.083235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.083509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.083779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.084468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.084810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.086198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.097421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 390/775] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.101673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.126838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.232173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.238971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829bd6b8-a33a-4200-8771-e349e7adea57 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.268103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52f1ae-52a2-430f-b646-edb549a7a61c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.303045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.304457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.304892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.305314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] PATH_INFO: `/v3/services/network` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.306049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.307804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.316272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.316932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.317220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.317499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.318208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] RBAC: Adding query filter params (interface=public, service_id=bae28f2be1f24d079900845cc7e301bc, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.318550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.319959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:30.324070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:30.325067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:30.325936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 386/776] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:30.327914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 391/777] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/endpoints?service_id=bae28f2be1f24d079900845cc7e301bc&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.331128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.333040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.441693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c6a31-38e2-41aa-818a-d3da7af18b58 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.478565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.479177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.479456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.479728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.480342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.481779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.485603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6bdb9-d3d6-4ea5-be92-560a84c98471 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.487084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 392/778] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.487804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.488258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.499858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.533332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.533969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.534250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.534676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.535272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] RBAC: Adding query filter params (name=network) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.535584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.539733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bd8c1-d1d3-4f80-a3aa-43a5ad9d62bb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.545236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 387/779] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.547522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2880-f4a3-4010-bbd8-5bf9536c0d9f None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.547817 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2880-f4a3-4010-bbd8-5bf9536c0d9f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.548096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e2880-f4a3-4010-bbd8-5bf9536c0d9f None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.549973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 388/780] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:30.551202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e2880-f4a3-4010-bbd8-5bf9536c0d9f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.614812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795d475-3b2f-4710-b3e4-0755b9a95987 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.658249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.658977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.659188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.659459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.670633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9c4d4474-d190-4f83-90b9-07678bdb5cb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:30.675287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:30.676256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:30.677156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 393/781] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:30.677868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.677868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.678009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.690855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e2880-f4a3-4010-bbd8-5bf9536c0d9f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.698160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:30.699160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:30.728024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.728667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.728943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.729210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.729829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] RBAC: Adding query filter params (type=network) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.730132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.731529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.741183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 389/782] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.741183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.769701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.781035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:30.789755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 394/783] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:30.791148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.912256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d73c4c-8667-493c-aea9-20e2687c6587 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.912509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52d21c-6e7a-4b73-826c-e6d744f3c2ea None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:30.953208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.954690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.955142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.955564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.956421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] RBAC: Adding query filter params (interface=public, service_id=015511f603174943a859686a249ddc15, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.956900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.958451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:30.967158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 395/784] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/endpoints?service_id=015511f603174943a859686a249ddc15&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:30.978789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:30.979614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:30.981003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:30.981549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:30.985197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:30.991112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:30.993216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:30.995144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.011703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 390/785] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:31.018983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.091317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.161528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.163833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b73c6-f98a-47e3-9f0f-e5122b995fd1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.168180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a63ed-aa6e-4b52-a5ef-89f5893c9b05 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.204225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.204737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.205265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.205388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.205681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.205955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.206132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.206574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.207319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.209071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.217809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.218058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 396/786] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:30 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:31.219136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.219355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:31.235853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:31.237119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:31.238057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:31.238057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:31.238057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:31.238057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 391/787] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:31.243303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.350075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae96401e-6444-456a-90ef-084ecd08cd36 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.366253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44ef3e-cfe5-4dce-b68f-aae4afb70bbb None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.380431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.392219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.393285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.394866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.394866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.405853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.406804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.407273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.407749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.408111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:31.408808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:31.409289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.410900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692096e-cf81-4d73-bc5c-cc5e01afc41d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d071cbae-8d8d-4321-8669-507f24eea837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:31.413669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:31.414652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:31.415554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 397/788] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:31.422018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0c2f-57d7-4589-8c6e-a88d22f6d181 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.422018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0c2f-57d7-4589-8c6e-a88d22f6d181 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.422018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b0c2f-57d7-4589-8c6e-a88d22f6d181 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.422018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 398/789] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:31.422935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b0c2f-57d7-4589-8c6e-a88d22f6d181 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.437031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 392/790] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:31.438843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.439513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.440019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.454823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.463058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:31.469203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:31.537341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b0c2f-57d7-4589-8c6e-a88d22f6d181 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.550541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 393/791] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:31.551470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.576884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.577494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.577767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.578034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.580241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:31.580543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.581916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.598542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 399/792] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:31.606385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.708342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.750832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c8b10-4ba3-44a2-9d0e-edb5a7cf68c9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.755581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783c5a-4fd5-4734-a161-875d612d660c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.785542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.786179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.786482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.786756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.797611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.799136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:31.814636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:31.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:31.817059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:31.817059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:31.817059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:31.817059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 400/793] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:31.823381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.836059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.836977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.837404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.837822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.847534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.858440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.858440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:31.874672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:31.875892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:31.877077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:31.879020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 394/794] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:31.884367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:31.949743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:31.952801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924b752-9776-4658-8444-d4935e5fd8fa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:31.989802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:31.990625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:31.991525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:31.991873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:31.992651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:31.992965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:31.994558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.001079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a948ef-d823-4141-859a-4bf85def878e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.006972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 401/795] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:32.007876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.040276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.041436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.042265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.042999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.043972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:32.044462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.047268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.071490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 395/796] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:32.072878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.122287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eafde-369f-47b1-8468-b9f03c1b87b4 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.145793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.160216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.161302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.161302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.161518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.173075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:32.179498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.180448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:32.181315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:32.186392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 402/797] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.187321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.205449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800e5a5-871f-46cf-92c5-146ae56de181 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.211579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.251038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.252216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.252707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.253071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.263675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.266121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:32.281399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.282606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:32.283724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 396/798] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.284330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.307623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea53959-67f3-41e3-8ffc-aa63c73480b6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.350195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.351973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.352728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.353642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.368413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.371211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:32.388777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.389826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:32.391307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:32.391307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.391307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:32.394505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 403/799] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.395542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.409345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4a78b-06cd-43b0-bd7e-f3f001518642 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.425789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.448677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.449688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.450187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.450796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.451689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:32.452274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.454163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.464813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 397/800] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:32.466041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.473740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.505430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd361c-0e2f-4f90-a77e-17022a898945 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.543557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.544226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.544871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.544871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.545473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:32.545780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.547248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.565289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 404/801] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:32.566570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.583469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5debff4-c006-4eec-8ca5-fefc43c3cc91 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.621983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.623072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.623719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.624192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.625083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:32.625591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.627407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.652935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 398/802] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:32.654274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.674327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f3c-f07c-4af9-9f31-199895d405cd None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.693039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.703944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.710312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.711023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.711305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.711577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.721920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.723410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:32.741194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.744786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:32.745746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:32.745746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.745746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:32.747244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 405/803] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.748262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.774554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac066947-d9b7-46f5-9cb6-6b232ee80578 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.809258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.809894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.810174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.810471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.821756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.823257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:32.837646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.844374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:32.845302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:32.845302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:32.845302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:32.845302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 399/804] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.845302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.867091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31feb-db80-4e11-9156-1946a7d7805c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.904194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.904749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.905030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.905305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.912572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.914135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-334f1f07-2a80-40a0-9246-9fb532cb5efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:32.927140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:32.928827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:32.929758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 406/805] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:32.934473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7f3c7-0195-4802-8bf1-07c3f80fa4c8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.934761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7f3c7-0195-4802-8bf1-07c3f80fa4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.935032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7f3c7-0195-4802-8bf1-07c3f80fa4c8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.935932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.936434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 407/806] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:32.936971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7f3c7-0195-4802-8bf1-07c3f80fa4c8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:32.952917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510a1df-86af-472e-9780-d9bb8ea9c019 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:32.981613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:32.990164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:32.991340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:32.991616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:32.991890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:32.992540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:32.992863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:32.994231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.012038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 400/807] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:33.013025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.040616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7f3c7-0195-4802-8bf1-07c3f80fa4c8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.077336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.077966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.078211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.078519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.079157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:33.079455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.080918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8032ff-5c75-440a-9e6d-a267a3e5d00f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.098643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 408/808] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:33.098643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.098643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.098643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.114901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:33.116609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:33.125775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fa1a7-eec6-46c6-a3a1-2d3b1ba5eafd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.163344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.163871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.164146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.164421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.165107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:33.165412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.166892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.179630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 401/809] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:33.180411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.191551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 409/810] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:33.192388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.200052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.229205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.308348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10a7f2-a8b5-40e4-9e02-8237b5b38b12 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.334060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19e30a-5d27-4b5c-a82a-472ae17e7e97 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.352539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.353179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.353458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.353731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.361896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.363489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.374721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.375349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.375624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.375893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:33.375989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:33.377028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 402/811] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:33.377986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.384648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.386092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-6183be14-91de-4523-b0ac-1fd2fc49a255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:33.398217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:33.399373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:33.400289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 410/812] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:33.400779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700018b-9b19-4cd0-ac8a-27e09d568652 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.402446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700018b-9b19-4cd0-ac8a-27e09d568652 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.402446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700018b-9b19-4cd0-ac8a-27e09d568652 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.406580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 411/813] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:33.406580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700018b-9b19-4cd0-ac8a-27e09d568652 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.451045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.484852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d863d-6f3a-4b2e-9d2b-349dbd62d09a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.504576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.514332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700018b-9b19-4cd0-ac8a-27e09d568652 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.521585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.522249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.522562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.522843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/d3b59fdbefe24db892bb40b4b23fe78c/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.537448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=d3b59fdbefe24db892bb40b4b23fe78c, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.539433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c254ba-dbe2-4408-be45-6e0649ac8282 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.547959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.548564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.554544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.554544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.554544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:33.554544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.554544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.570057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 412/814] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:33.570896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.613922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 403/815] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:33 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/d3b59fdbefe24db892bb40b4b23fe78c/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:33.615284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.615570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.616461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.649952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:33.651245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:33.676220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a7b6-adc0-4e6f-a130-997ed24bbcd7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.714349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.715042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.715397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.715676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.716384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:33.716689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.718144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.730567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 404/816] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:33.731880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.738686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 413/817] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:33.738686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.755390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.797741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:33.852693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878c60d-59d7-4508-b700-450b4b17c691 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.859876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738d2ae-b2d2-41be-8b5f-c94e46d29868 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:33.894440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.895073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.895596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.895876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.897609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.899364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.901518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:33.902119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:33.902393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:33.902689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:33.909950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:33.911228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 05:11:33.912364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:33.916863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 405/818] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:33.916863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:33.979953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:33.981473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:33.993398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.040226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42cde0-25bb-499f-ac9e-b462e663d1d1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.053403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.077649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.078475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.078760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.079318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/01a4e7f4d1e54b17938c3514428a34b9/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.081743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=01a4e7f4d1e54b17938c3514428a34b9, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.084374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.090754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 414/819] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:11:34.090754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.166432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 406/820] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:33 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/01a4e7f4d1e54b17938c3514428a34b9/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:34.166432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.195873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa08250-8750-476a-aff7-e7a2f57dd223 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.208725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.236471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.237163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.237585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.237948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.238743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:34.239146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.240776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.250858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 415/821] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:34.251944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.301919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790204f-f15a-408e-8362-5932080716c9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.344521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.345165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.345609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.345887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] PATH_INFO: `/v3/services/network` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.346541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.347924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:34.359316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:34.360355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:34.361223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 407/822] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:34.362076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.373329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8428f28-91c6-4c27-91c6-f7878ed9dde7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.400468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.410667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.411496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.411919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.412335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.424258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.425809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:34.443911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:34.448261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:34.449192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:34.449192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:34.449192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:34.449876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 416/823] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:34.450498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.483302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.486301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04541cc6-14c7-47bb-a65c-6ded69457f19 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.524841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.525697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.526225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.526677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.527498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:34.527982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.529507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.540483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 408/824] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:34.541728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.558232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9207476-f732-4ccf-b99a-cb58c2b77aa6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.593147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.593759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.594035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.594305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.595268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:34.595268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.596614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.606378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 417/825] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:34.606378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.666871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1413a7f-cd1e-42d0-840d-4d31dcf2d86b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.694848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.704954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.705565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.705843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.706114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.706968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:34.707429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:34.708204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.710272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.716760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bf124-3cde-4667-8c88-30176b7096f2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.723002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 409/826] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:34.724298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.759062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.762222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.762222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.762222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.762222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.762576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.832365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24872-cffe-4b63-ad1f-493a396d5262 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:34.866104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:34.866812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:34.867018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:34.867284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:34.867910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] RBAC: Adding query filter params (type=network) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:34.868211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:34.869610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:34.881826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 418/827] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:34.881826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.892592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 410/828] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:34.898519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:34.945787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:35.079381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a0a29-867a-457f-9330-608bfaf1ae58 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:35.092008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56731b-7d0f-4ae4-b10d-1c89af3a4d67 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:35.135019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:35.135637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.135916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.136188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.136861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:35.138555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:35.139250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:35.140103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.140573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.141149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.151320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:35.153146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:35.167624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:35.168644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:35.169843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 419/829] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:35.175014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:35.177760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:35.232916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 411/830] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:11:35.311353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd5865c-6925-4023-bef6-53c181bbd833 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:35.367785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:35.368746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.369289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.369781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.370702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:35.371210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:35.372872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe1434-8053-407f-bf4e-098cd174213b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:35.391589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 420/831] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:35.397971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:35.513088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:35.535528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e56055-c3bb-4cc5-9109-7babfac283e1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:35.586423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:35.587681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.588253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.588877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.589874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:35.590839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:35.592436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18dd1ca-e02a-4247-993c-ce3b8af0f56c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:35.599125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 412/832] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:35 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:35.670589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04bdc2-97d2-456f-880b-1c7b8d8826c0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.670913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04bdc2-97d2-456f-880b-1c7b8d8826c0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.671135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04bdc2-97d2-456f-880b-1c7b8d8826c0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.672785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 421/833] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:35.679869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac261a-ab61-4d08-a879-afa9e46378ac None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:35.680187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac261a-ab61-4d08-a879-afa9e46378ac None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:35.680418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac261a-ab61-4d08-a879-afa9e46378ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:35.701437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3cac261a-ab61-4d08-a879-afa9e46378ac None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:35.702684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3cac261a-ab61-4d08-a879-afa9e46378ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:35.780734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 413/834] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:36.055079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04bdc2-97d2-456f-880b-1c7b8d8826c0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.117330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.160907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04bdc2-97d2-456f-880b-1c7b8d8826c0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.196337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.196974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.197250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.197517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.209085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e91-b5bb-47df-9347-76156e3bece9 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.210310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:36.210893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e91-b5bb-47df-9347-76156e3bece9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.211367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e91-b5bb-47df-9347-76156e3bece9 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.213276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 414/835] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:36.217298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:36.218634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:36.219690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 422/836] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:36.220560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.220560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.220560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.222115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.241942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:36.243791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:36.327870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa9266-4cf0-4b7f-8df7-1df13d5dd24d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.330940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 415/837] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:36.346145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.363130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.363739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.364018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.364289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.364954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:36.365256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:36.366680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:36.375019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 423/838] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:36.380361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.384477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.482706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bdd59-5d69-4b06-9590-8a08a2f8f25a None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.617262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.635403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.638486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.638486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.638486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.648218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:36.649790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:36.665245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:36.666234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:36.667203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:36.667203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:36.667203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:36.671106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 416/839] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:36.672141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.739244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.748502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6153a8-c807-426f-bf7b-07bfcfccc6ad None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.788852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342900a-308f-4f8c-a5b7-92cf7b278dd5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.793176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.793813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.794091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.794362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.807913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:36.814519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:36.815763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:36.816645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 424/840] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:36.822462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.829922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.830559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.830838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.831108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.831838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:36.832189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:36.837161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:36.854502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 417/841] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:36.855852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:36.879938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.953080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa80529-2037-4e7b-b214-0b469ae7211c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.962925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480946ea-8c73-474b-934d-163c0284ddf7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:36.981040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:36.995844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:36.996453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:36.996727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:36.996993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:36.997654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:36.997950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:36.999349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.003763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.005079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.005601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.006385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.012826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 425/842] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:37.014781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.015246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.016946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:37.032299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:37.033262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:37.034219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:37.040991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 418/843] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:37.043036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.128640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38be47-6abc-47ef-9006-093fe1a3c9b8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.130320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.161481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cb0e8-ffd6-402d-8a40-2faf97aade0a None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.164346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.164967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.165270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.165538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.176280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.177716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f70859bd-1730-4bb5-af6a-17fe35fe0a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:37.192010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:37.192982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:37.194437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:37.194437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:37.194437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:37.194437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 426/844] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:37.194945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad457b03-e620-40c3-b8a5-276e281a8874 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.195037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad457b03-e620-40c3-b8a5-276e281a8874 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.195313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad457b03-e620-40c3-b8a5-276e281a8874 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.196670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 427/845] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:37.201135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad457b03-e620-40c3-b8a5-276e281a8874 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.205930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.206578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.206859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.207132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.207873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:37.208179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.210250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf4323-cb41-4371-bfb9-ef681a7f97b0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.230077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 419/846] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:37.231453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.231734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.232004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.255538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:37.257844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:37.289696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.312361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad457b03-e620-40c3-b8a5-276e281a8874 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.330610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 420/847] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:37.331700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.358719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.359425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.359775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.360091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.360835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:37.361261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.362722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.380559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 428/848] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:37.381942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.399350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.451362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e4ebe-b635-4135-a165-7f654ef2bcfa None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.486091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.486824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.487209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.487463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.495906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052a59-ae1b-46f2-9bf5-2c7aa58e6734 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.519809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.533293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.534254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.534770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.535266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.543198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.544934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:37.557812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:37.558830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 05:11:37.559827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:37.560377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.560454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.562604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 429/849] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:37.567764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.649946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.670954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 421/850] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:11:37.671796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.675101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e576925-0756-4f8e-bee1-daa3d915da1e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.709003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.710133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.710429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.710707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.711435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:37.711738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.713170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039422c-a0d5-47d6-a288-fe55830411de None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.730259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 430/851] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:37.731531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddadee-88b3-4dfc-8deb-b79e818f97ee None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.731816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddadee-88b3-4dfc-8deb-b79e818f97ee None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.732088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ddadee-88b3-4dfc-8deb-b79e818f97ee None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.733274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 431/852] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:37.733967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddadee-88b3-4dfc-8deb-b79e818f97ee None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.737684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.776142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54cbfc-63fb-48c8-ac77-79add5c532c9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.809551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.810186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.810495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.810772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.811360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.812744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:37.821873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:37.822970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:37.823825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 422/853] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:37.824644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:37.840275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ddadee-88b3-4dfc-8deb-b79e818f97ee None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.870055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.874932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.875556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.875832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.876098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.886331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:37.887926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-583e23b4-0253-4cca-92f7-27f368dcc4a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:37.903274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:37.904285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:37.905208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:37.905208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:37.905208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:37.905208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 432/854] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:37.905807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.906092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.906366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.924378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:37.925558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:37.926605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052c4e2-e274-454b-9e1b-902de12735c0 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:37.951566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:37.959120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:37.960385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:37.960666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:37.965550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:37.994454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 433/855] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:37.994454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.033383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.035176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.102789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d96f2-984a-497d-9275-8150a4feeae2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.117078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.136797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.137383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.137833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.137833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.138764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.139063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.140417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.146124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 423/856] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:11:38.146124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.150035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 434/857] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.154661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.176845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.249388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e85dcd-3c0f-4eec-aea7-2ee03652a131 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.253936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a1064-93c7-4010-ae85-de28b4b5a7db None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.283840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.284437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.284713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.284983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.285627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.285923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.287901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.289045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.289671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.289946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.290214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.299121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 424/858] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.299931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.300421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.302027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:38.316713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:38.317809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:38.319189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:38.319189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:38.319189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:38.319189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 435/859] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:38.319189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.347456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.390200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.401877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d24ea2-93c6-4d3a-a84c-43be35bd2687 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.415099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafb0ab-2d80-4526-9d59-a5ea13b1d3ed None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.434543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.435145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.435429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.435700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.436322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.436626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.438002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.442260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 425/860] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.443137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.460355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.461010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.461286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.461552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.462145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.463605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:38.472883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:38.473939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:38.475852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 436/861] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:38.480406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.563439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffce92-089e-485b-b8e1-b8dc0a54c80e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.580075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.588360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b0476-1ed1-435f-8fee-36cd870b26a8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.604672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.605308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.605583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.605845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.612922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.614382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.624451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.625308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.625725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.626160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:38.626976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:38.628026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:38.628955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 426/862] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:38.629464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.629667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.629981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.633796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.650383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.651337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 437/863] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.654567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.743824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8040c-579c-46cd-abff-164862b97291 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.755512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e941f-465b-4ef4-8999-683615c1147c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.779624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.780378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.780754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.781142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.781840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.782211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.783674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.793163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 427/864] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.793163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.796368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.797046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.797389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.797665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.798359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.798705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.800118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.817557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 438/865] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.817557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.833513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.876009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:38.901729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d522b91-6d06-4769-9929-db3b853808ce None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.929459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ff543-2b42-46f3-9ca5-a17a5cf61b94 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:38.934995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.935691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.936029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.936361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.943230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.945450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:38.956688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:38.957756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:38.958887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:38.959769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 428/866] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:38.960434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:38.973469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:38.974080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:38.974359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:38.974694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:38.975327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:38.975627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:38.976987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:38.981955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 439/867] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:38.981955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:39.089705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:39.112277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f6dfd-9acf-4456-a527-1849f4520b74 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:39.112897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d310a41-40f8-405c-b7a8-16b66c0f4131 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:39.155678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:39.156306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:39.156486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:39.156763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:39.157456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:39.157760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:39.159196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:39.159335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418cd150-eee0-4084-a504-685408948fc6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:39.159794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:39.160074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:39.160340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:39.161089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=d3b59fdbefe24db892bb40b4b23fe78c) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:39.161771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:39.163395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:39.166787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:39.171537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 429/868] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/role_assignments?user.id=d3b59fdbefe24db892bb40b4b23fe78c&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:39.181677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 440/869] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:39.188355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:39.302581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efd021-9219-49eb-9abf-fbce93aa0dba None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:39.339967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:39.341000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:39.341484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:39.342005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:39.342776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=01a4e7f4d1e54b17938c3514428a34b9) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:39.343088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:39.344606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac41233-ab03-4561-aec1-6808d0e3c391 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:39.352449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 430/870] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:39 2022] GET /identity/v3/role_assignments?user.id=01a4e7f4d1e54b17938c3514428a34b9&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:40.372384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c706c-30b9-4f04-9481-247056bfd6f2 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:40.372950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c706c-30b9-4f04-9481-247056bfd6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:40.372950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c706c-30b9-4f04-9481-247056bfd6f2 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:40.374604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 441/871] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:40.379941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df023-befe-4c89-85de-f7760cf0060b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:40.380363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df023-befe-4c89-85de-f7760cf0060b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:40.380542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df023-befe-4c89-85de-f7760cf0060b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:40.399787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-508df023-befe-4c89-85de-f7760cf0060b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:40.400974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-508df023-befe-4c89-85de-f7760cf0060b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:40.462800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:40.472327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 431/872] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:40.738164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c706c-30b9-4f04-9481-247056bfd6f2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:40.842657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c706c-30b9-4f04-9481-247056bfd6f2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:40.877283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:40.877952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:40.878229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:40.878579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:40.879444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:40.881451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba2bb5-88ee-4fec-a834-41e34d524435 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:40.952864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:40.960026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 442/873] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:40 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:40.968655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530597ca-5972-4be1-a9d6-d0cda2329356 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:40.968948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530597ca-5972-4be1-a9d6-d0cda2329356 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:40.969284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530597ca-5972-4be1-a9d6-d0cda2329356 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:40.971054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 432/874] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:40.977225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8236-56d5-4751-b1e6-2b90cd09afe6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:40.977561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8236-56d5-4751-b1e6-2b90cd09afe6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:40.977782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da8236-56d5-4751-b1e6-2b90cd09afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:40.998646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b7da8236-56d5-4751-b1e6-2b90cd09afe6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:40.999905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b7da8236-56d5-4751-b1e6-2b90cd09afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:41.083667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 443/875] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:41.204221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a269ace-b364-4d2e-be3b-f9e8742d57c7 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.204726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a269ace-b364-4d2e-be3b-f9e8742d57c7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.205213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a269ace-b364-4d2e-be3b-f9e8742d57c7 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.207038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 433/876] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:41.215893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e377f-e0b6-41b1-8159-872c0d440395 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.216475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e377f-e0b6-41b1-8159-872c0d440395 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.216956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e377f-e0b6-41b1-8159-872c0d440395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.235575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2f3e377f-e0b6-41b1-8159-872c0d440395 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:41.237074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2f3e377f-e0b6-41b1-8159-872c0d440395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:41.306300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 444/877] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:41.354720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a269ace-b364-4d2e-be3b-f9e8742d57c7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:41.368618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211812-0842-40b8-bdc1-b53274f0e906 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.368984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211812-0842-40b8-bdc1-b53274f0e906 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.369369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211812-0842-40b8-bdc1-b53274f0e906 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.371228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 445/878] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:41.377710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.378730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.379207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.398638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:41.400198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:41.458926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:41.469711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a269ace-b364-4d2e-be3b-f9e8742d57c7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:41.470302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 446/879] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:41.505584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:41.506864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.507138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.507405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.508062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:41.509436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:41.587244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 434/880] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:41 2022] POST /identity/v3/services => generated 269 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:41.612129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:41.729565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8f4d9-5c8a-4c5b-b4b0-efe969167853 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:41.749666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:41.774117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:41.774772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.775047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.775313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.786127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:41.791406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:41.792364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:41.793353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 447/881] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:11:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:41.799140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:41.819803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:41.853487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:41.882907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f179c959-5e90-4fdc-b834-d58e2fa86a25 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:41.913249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf9495-87a4-41d8-b5e9-f48f68d09eb0 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:41.933250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:41.934024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.934400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.934788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.936005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:41.937786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:41.949750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:41.950830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:41.951684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 435/882] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:41.956110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:41.956966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:41.957455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:41.957931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:41.959384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:41.959762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:41.960104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:41.961617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:41.972448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 448/883] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:11:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:41.982360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.097322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ff086-268b-46fb-a071-c61ede4e8f0c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.120063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:42.120843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250dc924-0ce0-4405-8e38-13abd4c691e8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.123023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:42.139876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.140489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.140762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.141030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.141936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:42.142237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.143829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.150632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 436/884] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:42.155947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.173358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.174501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.175293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.175778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.191455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.193234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:42.211799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:42.212791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:42.213716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:42.213716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:42.213716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:42.215928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 449/885] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:42.221134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.283389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7da5-65c2-4c34-913e-c6c01705abeb None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.325854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4e56d8-2521-4541-b89c-e4ae61ad6b3b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.331722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.332329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.332608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.332874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.333503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:42.333829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.336303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.342974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 437/886] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:42.371104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.371741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.372019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.372292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.372990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:42.373295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.374841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.389943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 450/887] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:42.397113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.449021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:42.509738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e94de-fa81-422c-930a-1ae84f3a8637 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.544707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.545252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.545528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.545799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.553565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.555075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:42.566425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:42.567403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:42.568907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 438/888] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:42.575957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.593080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:42.683448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede79e1-64c5-430a-94d5-78a9381db7c8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.724939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.725542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.725817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.726085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.726843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:42.727144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.728563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe33728-a96e-44ad-ad57-eca94a577596 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:42.749094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 451/889] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:42.755062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:42.862296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206bbe6-30cc-467f-9a9e-b347563bea0b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:42.870188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:42.898256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:42.898947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:42.899230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:42.899503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:42.968679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:42.970155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.004592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712efc87-7ed3-4258-924d-58537084b80d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.004881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712efc87-7ed3-4258-924d-58537084b80d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.005156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712efc87-7ed3-4258-924d-58537084b80d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.007185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 452/890] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:43.016359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.016715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.017056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.038437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:43.039746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:43.080912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 439/891] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:42 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:11:43.088267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.094286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.120001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 453/892] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:43.193485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cf923-10a7-43de-a469-cc2d31447008 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.228049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.228674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.228945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.229213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.239287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.241460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:43.256474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:43.257434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:43.258328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:43.258328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:43.258328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:43.258328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 440/893] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:43.263779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.266592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.373411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc82da-19bc-40fd-ae20-17c164956973 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.385938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.415378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.416817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.417269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.417938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.419204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:43.419928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.422121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.437503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 454/894] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:43.443240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.463433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.519938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce2f26-562b-40cc-a7eb-02f81f0bd912 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.533614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.567461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.568898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.569633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.570128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.571066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.573179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.574162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bdfb-66e5-4654-9f11-b4772c85aeaa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:43.584866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:43.585892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:43.586777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:43.589949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 441/895] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:43.594465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.617317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.617952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.618217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.618514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.626639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.627991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-43acf6a9-61fc-453d-bb83-c1b4c1dabd56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:43.642392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:43.643337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:43.644309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 455/896] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:43.645286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c553-50d1-4cc4-887b-8c682b90efc5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.645720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c553-50d1-4cc4-887b-8c682b90efc5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.646186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6c553-50d1-4cc4-887b-8c682b90efc5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.650547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 456/897] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:43.650547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6c553-50d1-4cc4-887b-8c682b90efc5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.716935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9c843-77c4-401b-89e6-6577f32d964c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.739400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.751680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.752318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.754914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.754914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.754914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:43.754914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.755415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3655-f464-42d9-9976-11a3582fba17 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.760738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 442/898] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:43.761903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.762174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.762560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.781403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:43.782610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:43.784668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6c553-50d1-4cc4-887b-8c682b90efc5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.810646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:43.822877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:43.823491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:43.829516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:43.829516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:43.829516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:43.829516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:43.829516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:43.843999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 457/899] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:43.843999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.859246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 443/900] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:43.867413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:43.953031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540a3ef-a0a0-48ac-b8df-cb7031783c28 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:43.994021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7a4e3-9d9b-4a14-aede-b6dac571bce4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.000197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.001178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.002168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.002673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.003960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.004451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.006782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.011147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.014876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 458/901] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:44.020834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.032966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.033827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.034279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.034761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.035655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] RBAC: Adding query filter params (role.id=f936b1077a064f609d250871fbbd77f3, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=d3b59fdbefe24db892bb40b4b23fe78c) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.036154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.038371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.045460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 444/902] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:43 2022] GET /identity/v3/role_assignments?user.id=d3b59fdbefe24db892bb40b4b23fe78c&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=f936b1077a064f609d250871fbbd77f3 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:44.085221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.139575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.145425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84d00e-a826-45a2-b7b4-669b0eb47c25 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.184926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.185564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.185840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.186111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.186859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] RBAC: Adding query filter params (interface=public, service_id=90cb73cd33df4b1c9e1bd1775d517f4f, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.187166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.188586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.198240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 459/903] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/endpoints?service_id=90cb73cd33df4b1c9e1bd1775d517f4f&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:44.204812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.273640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad70c6-0f01-4637-ad74-18790ab68203 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.320689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.322000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.322747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.323255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.324200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.326352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.327340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a651e92-979e-4c99-91b4-e7cb832535f1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-78df7897-83b9-4f3d-b8db-81e933ea4d79 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:44.337645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:44.338686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:44.339587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 445/904] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:44.340385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262ef0-ffe0-45e2-98e4-b563b4c312ad None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.341298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262ef0-ffe0-45e2-98e4-b563b4c312ad None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.341298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed262ef0-ffe0-45e2-98e4-b563b4c312ad None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.342111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 446/905] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:44.344039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.346802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed262ef0-ffe0-45e2-98e4-b563b4c312ad None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.369199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.370214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.371346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.371823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.372876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.375335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071e2c-b721-4517-a42b-d7e9d6083954 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.390704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 460/906] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:11:44.390704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.390704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.390704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.405608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.410114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:44.411390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:44.472830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed262ef0-ffe0-45e2-98e4-b563b4c312ad None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.502575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 461/907] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:44.521903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.522613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.526434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.526434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.526434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.526434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.526434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.531459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 447/908] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:44.537674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.651006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a77dc1-6653-49f2-bf96-c09eb81213a0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.654136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.688360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.689355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.689831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.690298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.691141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.691638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.694094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.699607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 462/909] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:44.705436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.797667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.813855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fb19f-98c9-4581-bc15-11698a59a677 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.824322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:44.852713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.854022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.854566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.855040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.856032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] RBAC: Adding query filter params (interface=public, service_id=8ed379f151b04aa59bf7babdf1d996cd, region_id=RegionOne) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:44.856672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.858468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.867464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 448/910] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/endpoints?service_id=8ed379f151b04aa59bf7babdf1d996cd&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:44.872643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:44.936827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e1245-2f3b-454d-8ccc-e8d24401e2c8 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:44.980922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:44.981590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:44.982222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:44.982536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:44.983463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:44.985121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dddaa-b48b-468f-b828-8186b43bdf0c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:44.992834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25056f-62b0-4528-944a-7a77fb1001eb None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:45.031380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:45.032054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:45.032351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:45.032608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:45.033215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:45.034630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c4ddf9-0f4e-4328-b9da-66c78bd12216 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:45.041041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 449/911] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:44 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:11:45.049859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 463/912] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:44 2022] POST /identity/v3/services => generated 241 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:11:46.035889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad59c-250f-4bf3-a9fa-c12e8017afa9 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.036406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad59c-250f-4bf3-a9fa-c12e8017afa9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.036406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ad59c-250f-4bf3-a9fa-c12e8017afa9 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.038465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 450/913] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:46.043797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e758-0aab-492c-8a7c-505dea79c3b2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.044235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e758-0aab-492c-8a7c-505dea79c3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.044341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4e758-0aab-492c-8a7c-505dea79c3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.063253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-80f4e758-0aab-492c-8a7c-505dea79c3b2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:46.064448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-80f4e758-0aab-492c-8a7c-505dea79c3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:46.088150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:46.142545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 464/914] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:46.408560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ad59c-250f-4bf3-a9fa-c12e8017afa9 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.413536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcfe4-8d2c-4315-8d14-fdde85f04835 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.413816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcfe4-8d2c-4315-8d14-fdde85f04835 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.414085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebcfe4-8d2c-4315-8d14-fdde85f04835 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.416460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 465/915] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:46.420624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.420904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.421176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.440017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:46.441206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:46.451578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:46.520485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 466/916] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:46.522736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ad59c-250f-4bf3-a9fa-c12e8017afa9 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.557475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.558103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.558378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.558671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.569680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:46.574437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:46.575376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:46.576235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:46.579573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 451/917] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:46.581854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.668489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:46.684117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62972090-e364-4830-9120-cc9e15fdfcd8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.720333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.720940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.721215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.721486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.722150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:46.722468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:46.723836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:46.732958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 467/918] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:46.738440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.811636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.837504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:46.843572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fa41d-e258-4d56-b3c9-675f16d073ac None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.877651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.878395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.878794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.879133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.893210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:46.894820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:46.910266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:46.911339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:46.912320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:46.912320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:46.912320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:46.916347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 452/919] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:46.917675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.921232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e3eb1-1b4e-40ae-9c54-4405a14b1d05 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:46.957549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:46.958192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:46.958549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:46.959378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:46.960076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:46.961472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:46.970998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:46.972032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 05:11:46.972913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:46.977185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 468/920] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:46.978163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:46.991613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.024521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff9d23-502a-46a5-9b69-72b394f0be68 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.059464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.060110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.060330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.060599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.061243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:47.061547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.063691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc85da-23b6-4a0b-9a56-340a2f8bcdec None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.080972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 453/921] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:47.081228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ff1aa-05c0-47fc-9912-374a0d255c19 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.081636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f85e-0754-4f2d-afe5-36bfa3d8e2d0 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.081636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f85e-0754-4f2d-afe5-36bfa3d8e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.081913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f85e-0754-4f2d-afe5-36bfa3d8e2d0 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.084593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 454/922] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:47.084853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc66b1f-9469-46f2-9229-40e41dc6e990 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.085106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc66b1f-9469-46f2-9229-40e41dc6e990 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.085378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc66b1f-9469-46f2-9229-40e41dc6e990 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.086709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 455/923] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:47.090138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc66b1f-9469-46f2-9229-40e41dc6e990 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.097894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.117504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.118130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.118394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.118695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.119318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:47.119613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.120965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008caafa-47a4-4586-97ea-fc077eaade89 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.125483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 469/924] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:47.126617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cb31-51c6-4b04-b5fb-ac68946f8034 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.126894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cb31-51c6-4b04-b5fb-ac68946f8034 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.127159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192cb31-51c6-4b04-b5fb-ac68946f8034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.145595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3192cb31-51c6-4b04-b5fb-ac68946f8034 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:47.147353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3192cb31-51c6-4b04-b5fb-ac68946f8034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:47.192130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc66b1f-9469-46f2-9229-40e41dc6e990 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.211746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 470/925] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:47.212835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.213114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.213381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.220064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.228846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.234905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.234905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.234905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.235128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:47.235128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:47.237191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.238684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:47.251020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:47.252172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:47.253316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:47.257597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 456/926] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:47.258451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.306300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 471/927] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:47.307490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.334281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.371022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683930a4-7a81-4353-bab6-b151874c8378 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.410303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.411231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.411691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.412113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.413098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:47.413583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.415301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.421369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 457/928] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:47.426756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff795d0-8f7b-464c-98ac-8821f239dce0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.428506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.464805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.465460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.465739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.466010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.467122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:47.467427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.469293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.471730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.491008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 472/929] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:47.491887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.557417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782e49-d304-499d-8448-b02aef16fedf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.589982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.591687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e9ad7b-790a-4cc6-bb9d-83facf501fe6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.594776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.595373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.595646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.595914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.596568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.597933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.625903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.626587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.626861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.627130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.627703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.629061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:47.637904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:47.639124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:47.640189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 473/930] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:47.641203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.677951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 458/931] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:47.677951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.704271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.758633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5c1ba-d8b8-4a45-b30f-d9e81e2e1950 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.801824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3a9b7-fe53-4560-9913-9f89c8bef9d2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.804720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.805363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.805636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.805903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.817531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.818767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:47.836289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:47.837467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:47.839208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:47.839208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:47.839208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:47.839208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 474/932] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:47.839208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.850575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.851379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.851604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.851870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.853238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.853891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:47.862891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:47.863850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:47.865354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 459/933] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:47.866189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:47.922715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.953106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad331f47-cb22-4541-94a2-ec51cfb65dc1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.956442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:47.982116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1876d-82c4-4e0c-9eb1-0e63b50802c1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:47.989573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:47.990322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:47.990575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:47.991074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:47.991704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:47.992002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:47.993367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.002498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 475/934] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.002498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.020850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.022028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.022364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.022659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.023302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.023599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.024978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.042242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 460/935] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.042242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.111659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039dbfe7-8b59-4693-8fd6-2a9463c7a2ee None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.147883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.148408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6ce2f-b0ba-4009-ba2f-0fe505f43178 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.148862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.148862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.149025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.149628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.149935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.151327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.156166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 476/936] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.156944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.164588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:48.189433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.190038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.190312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.190607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.191229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.191528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.192903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:48.193017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.198392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 461/937] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.199581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.273616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0ee55-5908-46b8-bbd8-e29f532d1be2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.302360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cbf61-3310-4845-98c4-90581f39789b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.311497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.312129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.312402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.312672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.320097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.321534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:48.333352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:48.334925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 477/938] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:48.335950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.343810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.344422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.344701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.344973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.345645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.345952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.347368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.352513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 462/939] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.353302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.390177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:48.438091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:48.447657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89505f-b88d-4d6e-afdf-0015d48e6c99 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.455421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ae7a-4fab-4b0d-b76b-312e3225551b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.488485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.489816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.490097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.490343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.490568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.490972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.491078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] RBAC: Adding query filter params (interface=public, service_id=ba960611384b4e99a9a92654f1aeec02, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.491250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.491403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.491558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.492243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:48.492548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.492748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.493974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.502726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 478/940] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/endpoints?service_id=ba960611384b4e99a9a92654f1aeec02&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.502726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.513647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 463/941] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:48.513647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.603161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:48.608860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d842b6-12b8-4920-a4da-a707f21dfdf7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.610251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec92a43-1893-45c3-995a-c1937102e788 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.645672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.646851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.647126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.647389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.647994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.649372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.653352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.654152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.654618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.655054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.655304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 464/942] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:48 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:11:48.656309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.656416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:48.658063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b00db-56fc-4549-817f-d58515561004 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.753008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 479/943] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:48.791348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669c9b6-4bc4-4c4a-9f2d-29dca6681266 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:48.845432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:48.846230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:48.846542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:48.846814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/d3b59fdbefe24db892bb40b4b23fe78c/roles/f936b1077a064f609d250871fbbd77f3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:48.849060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=d3b59fdbefe24db892bb40b4b23fe78c, role_id=f936b1077a064f609d250871fbbd77f3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:48.850699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40574e12-5461-407c-ad68-e9e71b4c4d4c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:48.918571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 465/944] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:11:48 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/d3b59fdbefe24db892bb40b4b23fe78c/roles/f936b1077a064f609d250871fbbd77f3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:11:49.731258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5a7db-0643-4b10-9ed2-9180c824bf52 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:49.731805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5a7db-0643-4b10-9ed2-9180c824bf52 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:49.731805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5a7db-0643-4b10-9ed2-9180c824bf52 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:49.733263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 480/945] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:49.742052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:49.742586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:49.742668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:49.760697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:49.761873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:49.774083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:49.858440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 466/946] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:50.181321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5a7db-0643-4b10-9ed2-9180c824bf52 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:50.197396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:50.287213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5a7db-0643-4b10-9ed2-9180c824bf52 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:50.322990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:50.323603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.323880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.324153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.324812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:50.326254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:50.328947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING keystone.server.flask.common [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 05:11:50.368355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.sql.core [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:11:50.368355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 05:11:50.368355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [parameters: {'id': 'b3132ec10310492dac623bdbf5b3c3dc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 05:11:50.368355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81569) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3132ec10310492dac623bdbf5b3c3dc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3132ec10310492dac623bdbf5b3c3dc. Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:11:50.386546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 05:11:50.388002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application yield resource Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise self.value Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application session.commit() Sep 01 05:11:50.388999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:11:50.389913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:11:50.390874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:11:50.391706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [parameters: {'id': 'b3132ec10310492dac623bdbf5b3c3dc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:50.392593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:50.393411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3132ec10310492dac623bdbf5b3c3dc. Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:50.394330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 481/947] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/projects => generated 262 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:50.400897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:50.500336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae232a-b6fb-4be2-812f-21aec6d2f1b7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:50.536236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:50.537392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.537667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.537937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.544983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:50.546554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1a06afe8-05da-4783-8c9a-5de039ce5b2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:50.558255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:50.560326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:50.562543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:50.563571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 467/948] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:50.566527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:50.650622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:50.690434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfee71-cd47-4c67-ab83-e3bc3af0ecc6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:50.727328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:50.728332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.728832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.729322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.730196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:50.730749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:50.732599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8da358-46d8-4d11-8a27-c629805403fc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:50.754861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 482/949] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:50.781202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8e349-5595-4460-a129-2f7e1a10f9b2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.781749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8e349-5595-4460-a129-2f7e1a10f9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.781749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8e349-5595-4460-a129-2f7e1a10f9b2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.783465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 468/950] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:50.792459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638feaf5-bb96-46db-be41-69c08a39d7fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.793020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638feaf5-bb96-46db-be41-69c08a39d7fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.793457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638feaf5-bb96-46db-be41-69c08a39d7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.817063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-638feaf5-bb96-46db-be41-69c08a39d7fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:50.819184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-638feaf5-bb96-46db-be41-69c08a39d7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:50.827594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0701a7-f4e1-4397-8ab2-2dd35f9bd129 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.828091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0701a7-f4e1-4397-8ab2-2dd35f9bd129 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.828580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0701a7-f4e1-4397-8ab2-2dd35f9bd129 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.831184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 469/951] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:50.837582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b9aa-b4cf-491c-a60e-5fa75a850bbe None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.837863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b9aa-b4cf-491c-a60e-5fa75a850bbe None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.838122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b9aa-b4cf-491c-a60e-5fa75a850bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.854783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:50.857381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-92b3b9aa-b4cf-491c-a60e-5fa75a850bbe None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:50.858634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-92b3b9aa-b4cf-491c-a60e-5fa75a850bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:50.904698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 483/952] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:50.931940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 470/953] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:50.968275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf7631-0eaf-49ef-a44f-756d755f999e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.968891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf7631-0eaf-49ef-a44f-756d755f999e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.969377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf7631-0eaf-49ef-a44f-756d755f999e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.971155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 484/954] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:50.976945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:50.977235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:50.977511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:50.996427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:50.997787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:51.068964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 471/955] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:51.174549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf7631-0eaf-49ef-a44f-756d755f999e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.203369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.243749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.284315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.292668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf7631-0eaf-49ef-a44f-756d755f999e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.328015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235c57c-b5a3-4039-9b54-d520900d4fe6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.329715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.330516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.330891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.331273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.331934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.333941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:51.343852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:51.345038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:51.346067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 485/956] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:51.346988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.371772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.372405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.372683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.372955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.373540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.374948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:51.384211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:51.385255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:51.391745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 472/957] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:51.392695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.464472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199fcf21-bbfe-4ec0-9a0b-b473228468ba None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.486689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.496485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426acae-49ef-4543-9552-0a71bbb42a32 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.503899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.503899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.503899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.503899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.512120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:11:51.517159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:51.518166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:51.519130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 486/958] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:51.519891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.538875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.545006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.558227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.558762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 473/959] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:51.558762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.627565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5a602-9dfb-427c-bf75-9561bfbf428b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.669821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a657299-b677-47af-870a-966bd5d6e351 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.675543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.676156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.676434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.677705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.678336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.678662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.680907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.691748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 487/960] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:51.691748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.706498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.707198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.707485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.707754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.708446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.708748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.710109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.718538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 474/961] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:51.719512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.748744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.777693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:51.802867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097e2ae-9dc0-4f0b-bf82-bc4e0f4b7730 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.825988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1d67c-d961-44ff-8fcb-90cce3128098 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:51.837221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.837839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.838116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.838386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.839045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.839354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.840721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.846257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 488/962] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:51.851787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.872036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:51.876494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:51.890527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 475/963] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:51.890527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:51.992875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7d7d7-911e-45c3-b94c-fac394213020 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.004883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:52.034732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.034980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d307a99-52e2-4d96-98aa-cbccddf6c23c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.035082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.035082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.035316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.055347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.057116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.061687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:52.074203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:52.082762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:52.083724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:52.083724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:52.083724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:52.083724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 489/964] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:52.083724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:52.084030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.084030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.084030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.084030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.084308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.085965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.170640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 476/965] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:52.201577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d635bc4-e07b-483e-8454-8bebfe37843c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.248775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.249823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.250301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.250836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.251751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:52.252284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.253882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.268568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 490/966] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:52.277156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:52.392660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3cf548-9e2a-4f96-875e-632ffcacc3d0 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.431644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.432196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.432471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.432737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.441172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.442700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:52.454010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:52.455066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:52.456077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 477/967] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:11:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:52.463611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:52.484802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:52.569210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0942-6d64-4d6a-8575-d569ce7051f7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.605659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.606648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.607142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.607616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.608578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:52.609097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.610854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d6e04-9686-4ca2-8db1-e4956a2185a6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.629077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 491/968] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 05:11:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:52.638612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:52.678137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:52.746887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56d4c2-87c3-4c46-bcda-e0ee28812f4e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:52.780819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:52.781425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.781700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.781968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.825794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186faa-207b-4eda-91f4-b2effd32bdc6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.826309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186faa-207b-4eda-91f4-b2effd32bdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.826309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3186faa-207b-4eda-91f4-b2effd32bdc6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.828023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 492/969] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:52.832766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:52.833052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:52.833901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:52.852327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:52.852689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:52.853662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:52.854176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:52.888687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:52.934234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 493/970] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:52.955287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 478/971] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:11:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:11:52.961450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:52.974437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.061358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378c67-f4c6-42ef-93b8-c5e00a4f4032 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.096433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.097343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.097798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.098286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.109793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.111733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:11:53.128943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:53.130523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:11:53.131457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:11:53.131457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:11:53.131457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.133319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 494/972] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:53.139241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.208303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.239164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca88bb-97e4-4984-af6c-118594cd70d5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.275350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.282782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.283398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.284207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.284490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.296766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:53.296766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.296766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.296766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.302893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 479/973] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:53.308382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.329076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db4fc5-0ec2-463d-85e6-7e666f392abe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.367818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.368673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.369283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.370869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.371758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.374043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.429760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.sql.core [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:11:53.429760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 05:11:53.429760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [parameters: {'user_id': '2115ab2d4ef541818895b3a85d339ecb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 05:11:53.429760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81569) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 05:11:53.430511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645f99-42bd-4bfa-a8b3-0e0030cbb913 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Sep 01 05:11:53.439221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:11:53.440342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application yield resource Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application next(self.gen) Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise self.value Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application session.commit() Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 05:11:53.441678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.session.flush() Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application compat.raise_( Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 05:11:53.443418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 05:11:53.445366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application conn.query(q) Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application result.read() Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 05:11:53.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [parameters: {'user_id': '2115ab2d4ef541818895b3a85d339ecb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:53.449989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 05:11:53.453160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 495/974] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 05:11:53 2022] POST /identity/v3/users => generated 160 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:53.456518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.484101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.485130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.485619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.486118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.494505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.496288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:11:53.509152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:53.510090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:11:53.511215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:53.513965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 480/975] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:53.519764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.580951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.587212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe494d5-547b-4733-a3b7-8172664cc556 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.613615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.631881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c84c92-e966-4b5f-9d99-57c1289c651d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.645374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.646074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.646334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.646910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.657037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.659185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.672450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.673495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.674123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.674673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:11:53.675302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:53.676262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:53.678539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 496/976] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:53.679187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:53.679187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.680568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.682388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.702509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 481/977] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:53.704639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:53.811869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad87b06-2bfd-49bb-91fd-d6465dd6b181 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.815785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012fff7-99cb-448b-9c6d-83a053f54429 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:53.833628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:53.855182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.856012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:53.856634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.856915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.857190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.857728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.857902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:53.858194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.858670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.859152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.859731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f07dd-ef64-47df-b670-8a94e25b1df2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.860352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] RBAC: Adding query filter params (role.id=f936b1077a064f609d250871fbbd77f3, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=d3b59fdbefe24db892bb40b4b23fe78c) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:53.861787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:53.863480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710efbd5-925e-481c-94b8-ce6decc470f6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:53.902926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 497/978] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:53.919927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 482/979] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:11:53 2022] GET /identity/v3/role_assignments?user.id=d3b59fdbefe24db892bb40b4b23fe78c&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=f936b1077a064f609d250871fbbd77f3 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:53.992441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1702-369f-4d17-acdb-88669be3a333 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:53.992779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1702-369f-4d17-acdb-88669be3a333 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:53.992997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1702-369f-4d17-acdb-88669be3a333 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:53.994613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 498/980] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:54.000561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59d13-b8f4-404f-9f33-a2126d19266b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.001199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59d13-b8f4-404f-9f33-a2126d19266b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.001700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e59d13-b8f4-404f-9f33-a2126d19266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.022799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-70e59d13-b8f4-404f-9f33-a2126d19266b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:54.024437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-70e59d13-b8f4-404f-9f33-a2126d19266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:54.100782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 483/981] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:54.226177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266f98-69f1-455e-bbf3-b8e40729d352 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.226554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266f98-69f1-455e-bbf3-b8e40729d352 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.226765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266f98-69f1-455e-bbf3-b8e40729d352 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.228121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 499/982] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:54.233306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.233594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.233868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.252164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:54.253364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:54.322820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 484/983] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:54.374802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266f98-69f1-455e-bbf3-b8e40729d352 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:54.416825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:54.478636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266f98-69f1-455e-bbf3-b8e40729d352 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:54.514870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:54.515470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.515746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.516012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.516649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:54.518022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:54.592784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:54.610440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 500/984] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:54 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:54.649968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:54.712180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b93a0e-2c20-448e-8d47-fea453908ad0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:54.758125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:54.758808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.759084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.759349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.759925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:54.761302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:54.770301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:54.771584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:54.772905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 485/985] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:54.777665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:54.874466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:54.878432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5dd80-8c14-4ea9-94b5-35f7c3552fe7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:54.913324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:54.914208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:54.915005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:54.915280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:54.915914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:54.916417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:54.917768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:54.922857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 501/986] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:54.928281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.027320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:55.030786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898607c6-c0b0-4943-8150-0c23cc5fcefd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.064730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.065332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.065606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.065872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.066515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:55.066815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.068709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.078840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 486/987] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:55.080100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.180256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73326913-b1c2-407d-b1fd-6edf486e8d62 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.214186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.214821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.215096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.215374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.216061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] RBAC: Adding query filter params (interface=admin, service_id=ba960611384b4e99a9a92654f1aeec02, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:55.216366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.217713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee99c9e-27e0-40bd-80e4-af3801a759e0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.222713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 502/988] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/endpoints?service_id=ba960611384b4e99a9a92654f1aeec02&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:55.228116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:55.325748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61b81b-af99-43e9-9718-0f0f608396c3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:55.358536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:55.359139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.359413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.359678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.360274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:55.361643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d954b-ad56-4ba7-b505-3c9f13df20bb None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:55.367909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 487/989] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:55 2022] GET /identity/v3/services => generated 1727 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:11:55.938000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad389e99-1b97-4373-8ae3-162479ebdecb None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.938546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad389e99-1b97-4373-8ae3-162479ebdecb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.938546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad389e99-1b97-4373-8ae3-162479ebdecb None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.939971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 503/990] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:55.945366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:55.945769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:55.945947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:55.964627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:55.965828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:55.989032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.034560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 488/991] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:56.307302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad389e99-1b97-4373-8ae3-162479ebdecb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.335546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.409545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad389e99-1b97-4373-8ae3-162479ebdecb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.444196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.444848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.445121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.445432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] PATH_INFO: `/v3/services/image` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.445974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.447590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-f162318a-6451-41f0-a376-a36d73f8517e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:56.457010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:56.458081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:56.463290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 504/992] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:56.464529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.563509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27182-fb4e-4652-adb2-bf87b8a7dbe0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.563697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27182-fb4e-4652-adb2-bf87b8a7dbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.563969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e27182-fb4e-4652-adb2-bf87b8a7dbe0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.565319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 505/993] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:11:56.566338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4a663-b62d-4e7c-a2a9-7a89ec7a6a7f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.570208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.570740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.571206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.591803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:56.593539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:56.603173 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.603776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.604399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.604686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.605320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:56.605625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.607333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.612797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 489/994] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:56.618892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.650020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:56.671889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 506/995] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:11:56.725470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da300cc-78e7-4ed7-80a3-e287b603f5d1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:56.759056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:56.759752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:56.760043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:56.760316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:56.760939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] RBAC: Adding query filter params (type=image) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:56.761249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:56.762928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:56.774477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 490/996] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:11:56.942023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:56.963213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:57.041132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5830909-87d0-4c7e-be8b-e464767b8341 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.074906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.075538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.075816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.076086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.076668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.078043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:57.087125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:57.088123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:11:57.089009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:11:57.090479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 507/997] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:11:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:11:57.094347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.180927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:57.199818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5967d-7ea5-4649-9eec-f70fe3714a39 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.236711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.237316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.237592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.237859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.238517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:57.238821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.240233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e1501-d310-44dd-850c-fbe20825601b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.245250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 491/998] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:57.250746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.297286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce642b23-7c45-4adb-9308-1279a0ef04f3 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.297624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce642b23-7c45-4adb-9308-1279a0ef04f3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.297846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce642b23-7c45-4adb-9308-1279a0ef04f3 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.299328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 492/999] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:57.304323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.304611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.304889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.323651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:57.324858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:57.338866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:57.357339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725fa02-d40f-4468-a902-ec9db65db3e5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.392963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.393404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 493/1000] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:57.394957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.395240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.396063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.396697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:57.396999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.398378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.403425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 508/1001] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:57.409025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.487564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:57.506329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558baf1-2311-4350-b6bc-5cff08497516 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.539775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.540370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.540650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.540903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.541590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] RBAC: Adding query filter params (interface=public, service_id=7886ea85b6bf4620ba863ed3f73724c6, region_id=RegionOne) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:57.541892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.543293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.548352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 494/1002] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/endpoints?service_id=7886ea85b6bf4620ba863ed3f73724c6&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:57.553654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.659150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.662534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d482b-4cb1-4ac1-b742-e22e2be38664 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.699324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.700167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.700724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.701003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.701618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.703141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.718436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 509/1003] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:11:57.773680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e4262-2347-4949-93bc-25d8f44ccd18 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.810468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.811111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.811386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.811652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.812242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.813722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:11:57.823954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:11:57.825059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:11:57.827398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 495/1004] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:11:57.831603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.849073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:57.939558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd59cb-bcc6-455a-af1f-cd53911b1bca None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:57.972828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:57.973441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:57.973715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:57.973982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:57.974629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:57.974931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:57.976279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:57.981089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 510/1005] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:57.986902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:57.998913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:58.086779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6564d-ffcc-4baf-b6bb-59e6c53cd834 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:58.120287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:58.120961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.121193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.121384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.122002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:11:58.122298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:58.123709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d58a-ca24-4a07-b4e0-6d57062568e9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:58.128660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 496/1006] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 05:11:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:11:58.134188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:58.216593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:58.239140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c3fa9-a61a-461f-ba38-e13c2b2bde4f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:58.273064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:58.273676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.273954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.274225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.274942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:58.276307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba919e-2182-411c-8dbe-69496543c8e1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:58.358515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 511/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:11:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:58.672741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c11-41e4-4325-98ed-54757bfbab62 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.673030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c11-41e4-4325-98ed-54757bfbab62 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.673310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74202c11-41e4-4325-98ed-54757bfbab62 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.674678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 497/1008] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:58.679901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65136-11c8-4798-8476-6f7421c81398 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:58.680187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65136-11c8-4798-8476-6f7421c81398 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:58.680459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65136-11c8-4798-8476-6f7421c81398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:58.698432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-29e65136-11c8-4798-8476-6f7421c81398 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:58.699620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-29e65136-11c8-4798-8476-6f7421c81398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:58.774063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 512/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:11:59.039162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74202c11-41e4-4325-98ed-54757bfbab62 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:11:59.112017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:11:59.141633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74202c11-41e4-4325-98ed-54757bfbab62 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:11:59.174801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685050dc-3670-4537-8562-4668a00a618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:11:59.175406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685050dc-3670-4537-8562-4668a00a618b None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.175677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685050dc-3670-4537-8562-4668a00a618b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.175941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685050dc-3670-4537-8562-4668a00a618b None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.177103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685050dc-3670-4537-8562-4668a00a618b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:11:59.178503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685050dc-3670-4537-8562-4668a00a618b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:11:59.267111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 498/1010] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:11:59 2022] POST /identity/v3/services => generated 242 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:11:59.644568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ebcd-fcb4-4480-8b9c-b19f17c60df8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.645006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ebcd-fcb4-4480-8b9c-b19f17c60df8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.645206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1ebcd-fcb4-4480-8b9c-b19f17c60df8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.646679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 513/1011] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:11:59.651667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:11:59.651961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:11:59.652240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:11:59.670705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:11:59.671910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:11:59.749164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 499/1012] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:11:59 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:00.030612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1ebcd-fcb4-4480-8b9c-b19f17c60df8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.092630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:00.132076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1ebcd-fcb4-4480-8b9c-b19f17c60df8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.167100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.167790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.168074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.168342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.168926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.170316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2788b4cc-f739-4839-affb-e04bc19743cf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:00.180901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:00.181938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 05:12:00.183233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:00.186568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 514/1013] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:00.188196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.252704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:00.265427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8e1b-56b1-4834-8f26-66a4025a342d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.265706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8e1b-56b1-4834-8f26-66a4025a342d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.265982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a8e1b-56b1-4834-8f26-66a4025a342d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.267433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 515/1014] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:00.272830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.273112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.273380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.297587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:00.297587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:00.297842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb68d55-dee8-417e-bb39-fcfc4eb9c3a0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.336667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.337647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.338164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.338669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.339518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:00.340100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.341680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:00.351883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 500/1015] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:12:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:00.352838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.370820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 516/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:12:00.455258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa52b3-11f8-41c3-b553-6da91f4aa59c None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.477540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:00.489031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.489634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.489911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.490790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.490905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:00.491152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.492494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:00.497266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 501/1017] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 05:12:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:00.503009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.559954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:00.604027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b117c-3145-4f73-814e-59ac17353577 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.644574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.645761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.645986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.646149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.646250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.646939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.648264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:00.744662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711692f0-68b3-46aa-8da8-0e9daf596337 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.758132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 517/1018] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:12:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:00.793719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.794363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.794706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.794979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.796137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.797598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74d0a3-c6b8-440b-b61a-5d40ff013cae None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:00.799419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 502/1019] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:00.805645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:00.915882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d68d8-cee0-489b-ace3-1f57ccbfe6d1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:00.930219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:00.952636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:00.953860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:00.953860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:00.953860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:00.955744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:00.956726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7e42e-4a6a-411c-bc4a-0a76a8161a01 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:01.027782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 518/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:12:00 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:01.219454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417561-a91f-477b-95c4-99945c090ad7 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.219962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417561-a91f-477b-95c4-99945c090ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.219962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417561-a91f-477b-95c4-99945c090ad7 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.221232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 503/1021] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:01.227247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.227533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.227804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.246897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:01.248073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:01.317539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 519/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:01.584413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6417561-a91f-477b-95c4-99945c090ad7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:01.640556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:01.691509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6417561-a91f-477b-95c4-99945c090ad7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:01.725832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:01.726589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.726871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.727145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.727787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:01.729225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:01.738218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:01.739571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:01.740431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 504/1023] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 05:12:01 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:01.747150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:01.841160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:01.846182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a6c44-a360-4825-8c6f-4f5ef2463a13 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:01.879980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:01.880589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:01.880865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:01.881136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:01.881761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] RBAC: Adding query filter params (name=image) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:01.882075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:01.883464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:01.889060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 520/1024] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:01.897608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:01.998276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ce8d3-f153-4cf7-b27d-e666dd1f88e5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:02.012737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:02.033305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:02.033908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.034187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.034481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.035108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] RBAC: Adding query filter params (type=image) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:02.035415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:02.036828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:02.047463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:02.048282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 505/1025] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:02.148206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d079289-7dfd-4ffe-80d3-a740eb514aec None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:02.182386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:02.183007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.183282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.183549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.184237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] RBAC: Adding query filter params (interface=public, service_id=a1597ddbc1be420e9d3e51e1f161d17b, region_id=RegionOne) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:02.184542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:02.185900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79297c-5f07-4897-bb29-8c0fc9f6c7c0 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:02.195841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 521/1026] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:12:02 2022] GET /identity/v3/endpoints?service_id=a1597ddbc1be420e9d3e51e1f161d17b&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:02.197081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:02.299558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a6b5f-8299-4b88-bac5-6615ea6f8e0f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:02.333043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:02.333650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.333925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.334195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.334824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:02.336276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c7d74-cf42-4793-a616-9c786b5f232e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:02.350548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 506/1027] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:12:02 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:12:02.815360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf14f8-90ca-46c7-8579-2908e3e308c7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.815879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf14f8-90ca-46c7-8579-2908e3e308c7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.815879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf14f8-90ca-46c7-8579-2908e3e308c7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.817466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 522/1028] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:02.822466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877ffeb-de74-47eb-877d-4972f1155f46 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.822973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877ffeb-de74-47eb-877d-4972f1155f46 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.823256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877ffeb-de74-47eb-877d-4972f1155f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.841270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-6877ffeb-de74-47eb-877d-4972f1155f46 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:02.842499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6877ffeb-de74-47eb-877d-4972f1155f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:02.880859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:02.912707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 507/1029] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:02.965696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8a31e-e1df-4d99-89db-b30863e1f7fe None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.966029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8a31e-e1df-4d99-89db-b30863e1f7fe None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.966246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8a31e-e1df-4d99-89db-b30863e1f7fe None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.967750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 523/1030] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:02.972662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:02.972953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:02.973783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:02.992210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:02.993386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:03.061148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 508/1031] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:03.179066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8a31e-e1df-4d99-89db-b30863e1f7fe None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.203287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:03.281144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8a31e-e1df-4d99-89db-b30863e1f7fe None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.315112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.315742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.316019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.316289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.326832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:03.330032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:03.331555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:03.332550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:03.333934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:03.338216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 524/1032] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:03.339091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.429831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:03.440072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37220f-2cfa-440b-a8af-af7192e09197 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.445385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:03.449518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188be4e-2eed-4837-8cce-786930554150 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.478875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.479664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.479833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.480087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.481184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:03.482664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:03.486503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 509/1033] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:03.489620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.489811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.490228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.490529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.490828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.491511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:03.491814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:03.493177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:03.501673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 525/1034] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:03.507522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.600281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd8631-6277-4a5b-9a72-1795bf576c81 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.612279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784006e5-7a49-4f36-b0af-436c0808d4fa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.641092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.641807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.642082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.642463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.643092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:03.645640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:03.653128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.653762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.654040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.654311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.665424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:03.668343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:03.669494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:03.681773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:03.682956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:03.684287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:03.684287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:03.684287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:03.684287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 526/1035] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:03.688597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.722788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 510/1036] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:03 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:03.792844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598383-7e9a-409e-bde4-b2e69a28e65d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:03.828938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:03.829801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:03.829953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:03.830193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:03.830888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:03.831194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:03.832631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:03.846221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 527/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:03.851961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:03.887144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:03.966515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e430a-2a49-47e1-a648-9f68d851e0dc None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.000916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.001555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.001834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.002104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.010577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.012316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:12:04.023581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:04.024570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:04.025538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 511/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:12:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:04.031651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.092036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.138204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f094c-b4ab-48a4-b4cc-38a0dbcd9aaa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.173040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.173647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.173925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.174194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.174956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:04.175261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.177309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbee34d-30e3-43c9-b18d-8fda8fa851e6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.199008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 528/1039] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:04.200648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.243428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538434-22e6-4472-bba8-bba6ee868e82 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.243618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538434-22e6-4472-bba8-bba6ee868e82 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.243883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88538434-22e6-4472-bba8-bba6ee868e82 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.245381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 529/1040] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:04.249995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.250260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.250600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.269019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:04.270318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:04.288341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.303418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f6ca50-0aac-4c6e-ab5f-1a51e57f566e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.338097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.338798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.339029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.339264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.341503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 530/1041] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:12:04.406584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.408043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.491048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.510785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 512/1042] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:12:04.512738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.558149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.608513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.612461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5c8dd-5265-40d2-9ccb-f6e715cf1e29 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.652289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.652985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.653269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.653542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.668144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.670452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:04.685741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:04.686802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:04.687746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:04.687746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:04.687746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:04.687746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 531/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:04.692384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.720702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1a5c8-c323-4720-8eda-d47c7ed39037 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.756938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.757572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.757956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.758232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] PATH_INFO: `/v3/services/image` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.758965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.760353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:04.770588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:04.771874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:04.772958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 513/1044] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:04.777419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.795246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.801188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712a3e-191d-411a-9f04-5b37628602d6 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.836519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:04.836718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.837232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.837510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.838135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.838918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:04.839227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.840615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.852095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 532/1045] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:04.857596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.906131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea6f80-a285-47bd-a50f-f42d080fd558 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:04.941091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:04.942400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:04.942576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:04.942750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:04.943340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:04.943700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:04.945006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:04.950118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 514/1046] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:04.955397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:04.966156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bbf65-382a-4fd7-ae39-9f15f92852a7 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:05.001623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:05.002391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.002813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.003088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.011333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:05.013745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:12:05.027761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:05.033639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 533/1047] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:05.034887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:05.053661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:05.067478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04025172-5188-4eb2-a670-40690e94f3c3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:05.081379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:05.102495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:05.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.103378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.103703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.104334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] RBAC: Adding query filter params (type=image) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:05.104640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:05.106014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:05.115334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 515/1048] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:05.116241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:05.146921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d909c2-873e-4626-8df9-074490dc8afe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:05.182779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:05.183529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.183891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.184169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.184860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:05.185165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:05.186624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:05.199614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 534/1049] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:05.205467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:05.222291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924161a5-d546-4909-832b-e6f7867b9960 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:05.259497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:05.260387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.260839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.261265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.262251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:05.263901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122fcd-63a4-478a-b423-55288f1444d1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:05.284203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:05.306600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e713-4c1b-43a2-bc4d-6a6778a5fb23 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:05.339088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:05.339693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.339970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.340244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.340963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] RBAC: Adding query filter params (role.id=f936b1077a064f609d250871fbbd77f3, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:05.341271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:05.342735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c7f8c-6012-4ac8-91d9-ee7b9d225705 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:05.347916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 535/1050] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:05 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=f936b1077a064f609d250871fbbd77f3 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:05.370099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 516/1051] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:12:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:12:05.726321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220033e5-e0b8-4260-a035-f5928b45f60a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.726668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220033e5-e0b8-4260-a035-f5928b45f60a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.726902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220033e5-e0b8-4260-a035-f5928b45f60a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.728389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 536/1052] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:05.733312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:05.733600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:05.733874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:05.752653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:05.753843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:05.780184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:05.827425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 517/1053] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:06.099399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220033e5-e0b8-4260-a035-f5928b45f60a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.198130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220033e5-e0b8-4260-a035-f5928b45f60a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.231860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.232474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.232748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.233017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.240415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.243732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:06.248598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:06.253935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:06.258718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 537/1054] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:06.259458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.359473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e5969-5893-4b35-95d0-2fc747732cc8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.393558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.394208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.394447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.394732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.395981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:06.396282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.397663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.410533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 518/1055] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:06.411850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.521034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8e685-01b8-4b47-a284-d2aa496fbb1a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.555793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.556422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.556694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.556962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] PATH_INFO: `/v3/users/f86dcd09f6134159bdd28d8249aeeeb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.569843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] RBAC: Authorizing `identity:get_user(user_id=f86dcd09f6134159bdd28d8249aeeeb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.571434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.574757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 538/1056] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/users/f86dcd09f6134159bdd28d8249aeeeb3 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:06.578885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.604582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.683935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97940649-9b3a-479f-9e68-be70a4b72674 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:06.717997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:06.718915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:06.719428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:06.719878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:06.787500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:06.789246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:06.811938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.893999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 519/1057] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:06.895264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:06.952755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:06.994920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d8ba3-218b-4204-8b07-773ebb5da45c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.028953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.029588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.029862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.030129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.031268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.032695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c1a94-0f51-493c-aab4-395213bb2da9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.039965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 539/1058] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:06 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:07.041457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.142082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4104-9115-43a9-89c5-51177471578f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.165242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.176979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.178988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.178988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.178988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.178988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894, user.id=f86dcd09f6134159bdd28d8249aeeeb3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:07.179266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.180566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98463187-af1b-453a-9f30-d29717846f4b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.186220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 520/1059] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/role_assignments?user.id=f86dcd09f6134159bdd28d8249aeeeb3&scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:07.259621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2888-d0cd-456d-ab4b-803cfc07edd5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.260180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2888-d0cd-456d-ab4b-803cfc07edd5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.260180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f2888-d0cd-456d-ab4b-803cfc07edd5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.261949 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 540/1060] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:07.267404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f122e-f004-49fc-9b23-30270c850303 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.267690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f122e-f004-49fc-9b23-30270c850303 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.267957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f122e-f004-49fc-9b23-30270c850303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.287740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-625f122e-f004-49fc-9b23-30270c850303 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:07.288977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-625f122e-f004-49fc-9b23-30270c850303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:07.331675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296f41-d1a3-4320-9166-272d12e27805 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.331975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296f41-d1a3-4320-9166-272d12e27805 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.332888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a296f41-d1a3-4320-9166-272d12e27805 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.334215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 541/1061] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:07.339549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.339828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.340102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.357841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:07.359054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:07.364425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 521/1062] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:07.428770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 542/1063] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:12:07.634107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f122e-f004-49fc-9b23-30270c850303 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.696614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.740662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f122e-f004-49fc-9b23-30270c850303 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.753614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.760022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:07.778818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.779448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.779725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.780010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.790859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:07.795600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:07.801958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:07.803012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:07.803810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6d67b-d928-414c-b871-a91ece431666 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.803907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 522/1064] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:07.803907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.841616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.842225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.842530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.842801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.909369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec71e-ff00-4968-8618-bbaffb440c9f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:07.913531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.915012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.947124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:07.947651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:07.947959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:07.948202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:07.948860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:07.949162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:07.950568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:07.959267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 523/1065] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:07.964904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:07.975558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.009252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.028495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 543/1066] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:12:08.030167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.073508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870793-e28b-4e10-80da-6b64722cf20e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.107386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.108015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.108289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.108954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.120851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.122390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:08.137432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:08.138472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:08.139404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:08.139404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.139404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:08.139833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d582e83-820f-4e1b-a957-669442bf2687 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.143427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 524/1067] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:08.144080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.177388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.178017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.178290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.178587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.189095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.190558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:08.208546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:08.209515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:08.215559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:08.215559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.215559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:08.215559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 544/1068] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:08.215986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.231313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.238908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.249762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da33e7e-e7d3-43b3-a0b0-c0773795c50d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.286697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.287304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.287580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.287866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.288477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.288777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.290154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.301534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 525/1069] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:08.307102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.334257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98808d6-07f5-4d3b-acaf-85ba4cc9fea9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.369512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.370231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.370549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.370945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.371582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.371886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.373244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.394434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 545/1070] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.405800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd09d59-4f6f-417f-93a2-5275edeb4725 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.443917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.444555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.444833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.445103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.453148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.454673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:12:08.468776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:08.469919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:08.471565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 526/1071] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:08.476459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.549132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.584962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533c48-f884-49f8-a864-82ec61839e72 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.622463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.622463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.622463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.622463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.623067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.623234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.624663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.644454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 546/1072] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:08.650583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.666747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.752638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc4263-68e3-4f08-9e62-5342e4f8bda2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.786032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.786697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.786975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.787246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.797343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.798838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:08.812487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:08.813436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:08.814637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:08.814637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:08.814637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:08.814637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 527/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:08.819941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:08.916181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f1f8f-02f5-4f74-81a6-fc0b4645c1a2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:08.941259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:08.950159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:08.950789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:08.951065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:08.951333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:08.951964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:08.952264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:08.953625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322862e-a911-40e2-904e-e17d2f696451 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:08.964423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 547/1074] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:08.970217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.030230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.067248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd8d5c-1ae5-40f4-9667-7c7d9e007148 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.067686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd8d5c-1ae5-40f4-9667-7c7d9e007148 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.067960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd8d5c-1ae5-40f4-9667-7c7d9e007148 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.068365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4e3ec-742a-4706-b5e4-4a9d2ad4a042 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.069297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 548/1075] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:12:09.074196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.074515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.074793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.092233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:09.093408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:09.104691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.105518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.105938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.106350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.114522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.115976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:12:09.126965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:09.128050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:09.130062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 528/1076] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:09.134223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.162956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 549/1077] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:12:09.229445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49471541-9bbf-4d3f-ade4-c14778ab33f7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.250887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.262597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.263193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.263472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.263749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.264434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:09.264739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.266164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.277895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 529/1078] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:09.283835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.379023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf513e-50f9-4dce-90b1-fef237831f52 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.394641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.411711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.412358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.412631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.412899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/f936b1077a064f609d250871fbbd77f3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.415015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=a81fbda6880a41be8922a2f8978e7f21, role_id=f936b1077a064f609d250871fbbd77f3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.416472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.430488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.526091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 550/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:09 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/a81fbda6880a41be8922a2f8978e7f21/roles/f936b1077a064f609d250871fbbd77f3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:09.526345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8d69-af63-4dd4-acee-5545390bd8b6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.563836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.564367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.564643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.564911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.576640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:09.581540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:09.582654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:09.583570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:09.584291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 530/1080] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:09.589015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.696373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f62b4-63f3-4280-b577-37cb26c671ca None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.729416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.730010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.730283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.730592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.731250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:09.731547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.732889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.740670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 551/1081] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:09.746770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.790363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:09.843571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5515e-4457-4165-a843-35cac6170cbf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:09.877634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:09.878260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:09.878571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:09.878847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] PATH_INFO: `/v3/users/f86dcd09f6134159bdd28d8249aeeeb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:09.879966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] RBAC: Authorizing `identity:get_user(user_id=f86dcd09f6134159bdd28d8249aeeeb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:09.881415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:09.883242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 531/1082] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/users/f86dcd09f6134159bdd28d8249aeeeb3 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:09.889312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:09.949426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:10.008269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ae71-dfa8-4210-a956-3db2096e2157 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:10.042479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:10.043112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.043390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.043664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.044779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:10.046241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c425b6-4847-4403-acf9-fc1c14c20a7b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:10.047913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 552/1083] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:09 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:10.053723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:10.151875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9883cc4-38ea-499f-b70e-ab1faaf6c78f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:10.161782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:10.184306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:10.184961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:10.185354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:10.185632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894/users/f86dcd09f6134159bdd28d8249aeeeb3/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:10.187546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] RBAC: Authorizing `identity:create_grant(project_id=f3a27db737bc4bdd8c7f6e78b1274894, user_id=f86dcd09f6134159bdd28d8249aeeeb3, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:10.189066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7256b-0c46-4c9b-8a69-3212ffbf728e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:10.258904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 532/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:10 2022] PUT /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894/users/f86dcd09f6134159bdd28d8249aeeeb3/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:11.407865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416a186-e65f-465a-ad0e-7f20ef7ee322 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.408431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416a186-e65f-465a-ad0e-7f20ef7ee322 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.408431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416a186-e65f-465a-ad0e-7f20ef7ee322 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.409744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 553/1085] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:11.415922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.416357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.416455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.434195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:11.435425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:11.499510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 533/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:11.768770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416a186-e65f-465a-ad0e-7f20ef7ee322 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:11.866805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416a186-e65f-465a-ad0e-7f20ef7ee322 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:11.873503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:11.901147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:11.901774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:11.902047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:11.902318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:11.912040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:11.916777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:11.917941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:11.918851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 554/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:11.923831 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.017004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.019939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b641670-6846-4ae2-90c8-3ff8b445c2fb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.052888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.053666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.053987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.054787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.055398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.055698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.057079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cd7ad-e399-403f-afe3-2c756d38e7f4 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.064687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 534/1088] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:12:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.071058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.123135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ccb9-fe3a-4853-b605-52f71222e408 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.123413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ccb9-fe3a-4853-b605-52f71222e408 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.123682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ccb9-fe3a-4853-b605-52f71222e408 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.125132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 535/1089] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:12.130238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.130547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.130818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.148715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:12.149926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:12.170347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfae3b-1785-454d-bd02-4e60a95be925 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.204088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.204713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.204994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.205270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.215604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.215745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 536/1090] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:12:12.217059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:12.230852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.231957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:12.233073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:12.233073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:12.233073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:12.234552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 555/1091] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:12.237659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.324212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.332901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb5ee-adac-4b35-b979-e9a19e29f275 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.364915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.365503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.365778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.366045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.366706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.367006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.368366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.379745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 537/1092] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.385244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.405787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.483675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3ed69-1c54-4f35-8ba4-a039a6376791 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.486217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.517317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.517944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.518227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.518537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.526477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.527912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:12:12.538688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.539670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:12.540676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 556/1093] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:12.545540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.589098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267207ab-a0b5-4ef8-8b1c-64345f5fc931 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.621308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.621930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.622204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.622500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.622603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.633675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:12.646340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:12.647317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 538/1094] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:12.648207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.650890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48378499-1b2a-4dd7-854c-cf56cc0ab19a None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.661217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.684457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.685058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.685331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.685594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.686347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.686672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.688079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.705553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 557/1095] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:12.711246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.744546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ad63-47fb-417e-818a-159bf6455111 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.778632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.779240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.779517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.779788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.780444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:12.780744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.782125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.791161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 539/1096] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:12.796646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:12.811244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87456e-42e3-4400-8622-d40081245368 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.844480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.845007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.845287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.845553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.849513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.868818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:12.893349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e9f23-3d63-45ad-adaf-b2d4c19b5c59 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:12.913212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.914696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.929254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:12.929876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:12.930151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:12.930428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] PATH_INFO: `/v3/users/f86dcd09f6134159bdd28d8249aeeeb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:12.931574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f86dcd09f6134159bdd28d8249aeeeb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:12.933019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:12.934616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 540/1097] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/users/f86dcd09f6134159bdd28d8249aeeeb3 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:12.940313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.012791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 558/1098] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:12:13.017430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.037473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac287f55-e2c6-460a-b981-d890ed00f5b3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.051385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.072110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.072713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.072994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.073300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.085489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.113947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52c07-4a03-43a1-ae06-b33d6ad62f2e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.141053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.142553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.149188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.149813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.150090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.150366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.160054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.161578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:13.175781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:13.176932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:12:13.177858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:13.177858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:12:13.177858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:13.177858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 559/1099] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:13.182469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.241599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 541/1100] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:12:13.248010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.268340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.280510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3b0e1-3fd4-4ed4-947f-e84f7a5b6c5d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.289768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.312996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.313604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.313886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.314161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.314854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.315154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.317028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.328015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 560/1101] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:13.334105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.344999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2510-0608-4af0-8a0d-105b539560c9 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.377472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.378279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.378583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.379345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.380450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.381884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.383522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 542/1102] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:13.389781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.429150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420e75d-5d66-4758-85d3-4c1dc34f585c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.461193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.461807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.462081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.462353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.468719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.470157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:12:13.481053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:13.481996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:13.483373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 561/1103] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:13.485949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426138ba-0b6b-4f0a-857d-dcccd00750bd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.487739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.498283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.499511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:13.520992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.521579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.521851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.522114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.522861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894, user.id=f86dcd09f6134159bdd28d8249aeeeb3) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.523161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.524585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.530124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 543/1104] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/role_assignments?user.id=f86dcd09f6134159bdd28d8249aeeeb3&scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:13.592189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df3e6-58fc-4b49-8885-b2bff1c256c2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.627054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.627725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.627946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.628215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.629469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.629773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.631269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49db-1692-4043-970f-c30c026625cd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.644805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 562/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:13.651001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:13.754453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af592a-7b47-4f44-bb42-59cf997f89cf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:13.788166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:13.788762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:13.789038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:13.789304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:13.790022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] RBAC: Adding query filter params (role.id=f936b1077a064f609d250871fbbd77f3, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=a81fbda6880a41be8922a2f8978e7f21) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:13.790322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:13.791778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b3a6b5-33c2-4442-be5a-aeee2a9cef3b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:13.797438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 544/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:13 2022] GET /identity/v3/role_assignments?user.id=a81fbda6880a41be8922a2f8978e7f21&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=f936b1077a064f609d250871fbbd77f3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:15.434937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad837b6-3b43-4505-8dd7-442b43d8bc79 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.435471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad837b6-3b43-4505-8dd7-442b43d8bc79 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.435471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad837b6-3b43-4505-8dd7-442b43d8bc79 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.436833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 563/1107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:15.442224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.442639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.442790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.460362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:15.461541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:15.490298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:15.525285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 545/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:15.792339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad837b6-3b43-4505-8dd7-442b43d8bc79 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:15.857871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:15.892387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad837b6-3b43-4505-8dd7-442b43d8bc79 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:15.926798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:15.927410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:15.927680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:15.927943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:15.932107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:15.932107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa36714-6d4b-4133-b9c5-be2111525272 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:15.932107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 564/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:15.937897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:16.031259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef3c85-87d0-4023-b154-af47b1639a93 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:16.064602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:16.065128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:16.065403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:16.065668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:16.066865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:16.068283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0266-c8dc-4bc0-befc-90debc49a995 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:16.178477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 546/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:15 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:18.087374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30f1d4-c4a6-481b-a3ad-a7c0b8be0092 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.087902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30f1d4-c4a6-481b-a3ad-a7c0b8be0092 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.087902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30f1d4-c4a6-481b-a3ad-a7c0b8be0092 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.089246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 565/1111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:18.094375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.094798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.094944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.113230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:18.114444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:18.130851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.179549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 547/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:18.444780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30f1d4-c4a6-481b-a3ad-a7c0b8be0092 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.541986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30f1d4-c4a6-481b-a3ad-a7c0b8be0092 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.574191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.574828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.575106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.575372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] PATH_INFO: `/v3/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.576759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] RBAC: Authorizing `identity:get_role(role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.578125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.579466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 566/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/roles/119f085c1d714b95aa1674da1868b4a6 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.586281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.687562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f97906-0f1e-4e04-9364-2a881ceaca21 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.720121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.720690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.720969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.721239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] PATH_INFO: `/v3/users/0ade89c1a5ce449797e7567ce043bc5e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.733573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] RBAC: Authorizing `identity:get_user(user_id=0ade89c1a5ce449797e7567ce043bc5e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.735063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.736637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 548/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/users/0ade89c1a5ce449797e7567ce043bc5e => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:18.742947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:18.746663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:18.839246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab40eb0-bc60-4e43-bf6b-b7649547508e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:18.872359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:18.872954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:18.873234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:18.873508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:18.943967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:18.945404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:18.948111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.045156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 567/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:19.051836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.066519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.147870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09f52e-810e-48a0-aebd-992e55ee66fd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.180812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.181429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.181707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.181974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.183088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.184559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b54a4-05ed-4681-bdcc-0d91bcc4409a None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.186165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 549/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:19.192285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:19.285976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16888a-d39c-4ac3-b9cf-2fae17208c0d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:19.288304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:19.318243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:19.318806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:19.319086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:19.319358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:19.320065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894, user.id=0ade89c1a5ce449797e7567ce043bc5e) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:19.320370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:19.321796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3dd34-1d83-4e88-b9ea-1eb3c3973a81 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:19.327131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 568/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:19 2022] GET /identity/v3/role_assignments?user.id=0ade89c1a5ce449797e7567ce043bc5e&scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.189801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550b4d-4e8b-4891-84ea-0355fe4ae112 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.190361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550b4d-4e8b-4891-84ea-0355fe4ae112 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.190361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550b4d-4e8b-4891-84ea-0355fe4ae112 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.192300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 550/1118] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:21.197387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.197798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.197936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.215838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:21.217003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:21.280093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 569/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:21.545718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47550b4d-4e8b-4891-84ea-0355fe4ae112 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.623794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.642677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47550b4d-4e8b-4891-84ea-0355fe4ae112 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.678767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.679416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.679695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.679967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] PATH_INFO: `/v3/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.681348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] RBAC: Authorizing `identity:get_role(role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.682759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.684083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 551/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/roles/119f085c1d714b95aa1674da1868b4a6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.689875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.781849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:21.785880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29890097-08bb-45fd-9f5e-2e7ab71e9b2e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.817450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.818184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.818713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.819002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] PATH_INFO: `/v3/users/0ade89c1a5ce449797e7567ce043bc5e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.820450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] RBAC: Authorizing `identity:get_user(user_id=0ade89c1a5ce449797e7567ce043bc5e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.821885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.823477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 570/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/users/0ade89c1a5ce449797e7567ce043bc5e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.829298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:21.924640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119b1901-4e90-4b0d-a546-8465af159f1f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:21.957160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:21.957764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:21.958041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:21.958310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:21.959444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:21.960876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae047d2-682e-4d6b-b4f2-4c26670d8e52 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:21.962483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 552/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:21 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:21.968108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:22.062930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253f61b-02aa-4b2e-b020-e3f81600198c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:22.095565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:22.096132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:22.096405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:22.096673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894/users/0ade89c1a5ce449797e7567ce043bc5e/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:22.098819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f3a27db737bc4bdd8c7f6e78b1274894, user_id=0ade89c1a5ce449797e7567ce043bc5e, role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:22.100223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55433ec-14fb-4302-820f-70424907d4e6 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:22.189035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 571/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:21 2022] PUT /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894/users/0ade89c1a5ce449797e7567ce043bc5e/roles/119f085c1d714b95aa1674da1868b4a6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:24.057078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af0b3e-7d83-4933-9dc8-03db084f8cfc None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.057611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af0b3e-7d83-4933-9dc8-03db084f8cfc None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.057611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af0b3e-7d83-4933-9dc8-03db084f8cfc None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.059017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 553/1124] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:24.064112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.064528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.064632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.082755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:24.083923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:24.102934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:24.147548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 572/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:24.418772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af0b3e-7d83-4933-9dc8-03db084f8cfc None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:24.436454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:24.517100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af0b3e-7d83-4933-9dc8-03db084f8cfc None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:24.549188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:24.549825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.550101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.550369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] PATH_INFO: `/v3/roles/119f085c1d714b95aa1674da1868b4a6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.551755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] RBAC: Authorizing `identity:get_role(role_id=119f085c1d714b95aa1674da1868b4a6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:24.553196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:24.554495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 554/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:24 2022] GET /identity/v3/roles/119f085c1d714b95aa1674da1868b4a6 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:24.560169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:24.654971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8714b-ecc7-4c33-bdb1-42c87535e116 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:24.687191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:24.687741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.688014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.688279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] PATH_INFO: `/v3/users/0ade89c1a5ce449797e7567ce043bc5e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.689372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] RBAC: Authorizing `identity:get_user(user_id=0ade89c1a5ce449797e7567ce043bc5e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:24.691410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:24.692946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 573/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:24 2022] GET /identity/v3/users/0ade89c1a5ce449797e7567ce043bc5e => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:24.698615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:24.778224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:24.792731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f408a-0efd-479d-a6f9-772bbdc8de2d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:24.825757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:24.826341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:24.826660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:24.826927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:24.890310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:24.891848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:24.968776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:24.983925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 555/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:24.990370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.015537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:25.086052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e36f2-3ebe-4daf-989e-ebc2e2a85fcc None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.118552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.119162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.119438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.119714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] PATH_INFO: `/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.120792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] RBAC: Authorizing `identity:get_project(project_id=f3a27db737bc4bdd8c7f6e78b1274894)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.122204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63056761-b90d-47b3-aec3-9290ace02baa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.123815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 574/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:24 2022] GET /identity/v3/projects/f3a27db737bc4bdd8c7f6e78b1274894 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:25.129791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:25.224125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dc989-5953-4d2d-9478-96e2191f84db None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:25.256211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:25.256790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:25.257063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:25.257330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:25.258036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] RBAC: Adding query filter params (role.id=119f085c1d714b95aa1674da1868b4a6, scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894, user.id=0ade89c1a5ce449797e7567ce043bc5e) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:25.258337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:25.259785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5475793-38bd-465b-9d64-434c2664a8ff None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:25.265573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 556/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:25 2022] GET /identity/v3/role_assignments?user.id=0ade89c1a5ce449797e7567ce043bc5e&scope.project.id=f3a27db737bc4bdd8c7f6e78b1274894&role.id=119f085c1d714b95aa1674da1868b4a6 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:27.181464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e53f2-40de-4e6e-a21c-7e7249e63917 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.182001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e53f2-40de-4e6e-a21c-7e7249e63917 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.182001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9e53f2-40de-4e6e-a21c-7e7249e63917 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.183375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 575/1131] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:27.188537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.188925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.189083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.207213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:27.208394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:27.249967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:27.272221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 557/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:27.547709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e53f2-40de-4e6e-a21c-7e7249e63917 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:27.638538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:27.648309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9e53f2-40de-4e6e-a21c-7e7249e63917 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:27.680760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:27.681376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.681653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.681923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.683032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:27.684463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52225d3f-f808-4e71-9858-3b9a9c5e6482 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:27.685995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 576/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:27.692373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:27.787922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca215a-e86d-41a2-8591-3f4a82d8855e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:27.821057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:27.821591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:27.821874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:27.822149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:27.822833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:27.824273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2718d5d-6bfe-41b5-b0bd-2c70ae0f0787 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:27.959924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 558/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:12:27 2022] POST /identity/v3/projects => generated 311 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:29.872241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabaf6-8d45-43d8-a380-2a238cbebb3f None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.872763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabaf6-8d45-43d8-a380-2a238cbebb3f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.872883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaabaf6-8d45-43d8-a380-2a238cbebb3f None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.874169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 577/1135] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:29.879960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483559f7-6e93-4111-8031-228e012dad8c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:29.882765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483559f7-6e93-4111-8031-228e012dad8c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:29.882765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483559f7-6e93-4111-8031-228e012dad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:29.899443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-483559f7-6e93-4111-8031-228e012dad8c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:29.900645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-483559f7-6e93-4111-8031-228e012dad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:29.923284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:29.965715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 559/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:30.232590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaabaf6-8d45-43d8-a380-2a238cbebb3f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.330145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaabaf6-8d45-43d8-a380-2a238cbebb3f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.362445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.363071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.363346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.363613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.364701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.366124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac18ac-b446-478a-b09d-11ff23d2aa69 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.367687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 578/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:12:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:30.373562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483559f7-6e93-4111-8031-228e012dad8c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:30.469540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483559f7-6e93-4111-8031-228e012dad8c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:30.502176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:30.502736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:30.503013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:30.503303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:30.503954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:30.505381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf9bf0-68fc-456e-aae0-1e07984ed08c None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:30.599303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 560/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:30 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:32.500260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a779d1e-23c4-45f8-9a09-2463106cfbc8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.500827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a779d1e-23c4-45f8-9a09-2463106cfbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.500827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a779d1e-23c4-45f8-9a09-2463106cfbc8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.502119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 579/1139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:32.507232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.507654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.507777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.526262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:32.527475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:32.538601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.592014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 561/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:32.855053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a779d1e-23c4-45f8-9a09-2463106cfbc8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:32.882503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:32.954353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a779d1e-23c4-45f8-9a09-2463106cfbc8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:32.985709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:32.986322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:32.987212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:32.987480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:32.996979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:33.001651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:33.003530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:33.004769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 580/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:33.009306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.103224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccdd16-82ec-422d-883f-4e7e6aef006d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.136299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.136827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.137102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.137372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.138026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:33.138327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.139726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.147415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 562/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.152855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.235773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.247579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daea2-e1b9-4b82-a9b8-1e99fad817f2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.286931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.287551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.287829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.288098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] PATH_INFO: `/v3/users/5ffbc8b6acc747928edb425b7c6d5790` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.301358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] RBAC: Authorizing `identity:get_user(user_id=5ffbc8b6acc747928edb425b7c6d5790)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.302824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.304410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 581/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/users/5ffbc8b6acc747928edb425b7c6d5790 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.312086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.323241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.407864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c778b-ab04-4dca-b11d-ca5bde1362f2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.440188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.440774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.441051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.441320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.505662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.507146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.516533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.612984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 563/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:33.619161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.720312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696456-88ff-4261-b83c-1214f14393e5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.752721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.753348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.753629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.753994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] PATH_INFO: `/v3/projects/47f8e1df43d843739d22a88d09254de3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.755133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] RBAC: Authorizing `identity:get_project(project_id=47f8e1df43d843739d22a88d09254de3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.756563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea831c2-b1cf-4c8a-9aad-9f06a69f1df5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.758178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 582/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/projects/47f8e1df43d843739d22a88d09254de3 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:33.764637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:33.861160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9739e35-066f-4013-9780-fb2bbe82761e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:33.869970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:33.893416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:33.894004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:33.894779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:33.895062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:33.895778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=47f8e1df43d843739d22a88d09254de3, user.id=5ffbc8b6acc747928edb425b7c6d5790) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:33.896084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:33.897509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fef5cac-6bfb-44a8-a53d-748b2381a041 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:33.902666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 564/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:33 2022] GET /identity/v3/role_assignments?user.id=5ffbc8b6acc747928edb425b7c6d5790&scope.project.id=47f8e1df43d843739d22a88d09254de3&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:35.740376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99905-3ddc-4018-b964-7b4731602560 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.740904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99905-3ddc-4018-b964-7b4731602560 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.740904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e99905-3ddc-4018-b964-7b4731602560 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.742220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 583/1147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:35.747542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:35.747922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:35.748091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:35.766027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:35.767246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:35.830940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 565/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:36.100718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99905-3ddc-4018-b964-7b4731602560 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.116059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.198108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e99905-3ddc-4018-b964-7b4731602560 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.232143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.232765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.233042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.233314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.243106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:36.247859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:36.248981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:36.250242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 584/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:36.255344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.349910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f21ee-f0a5-4bce-95b1-cf31d0e9ba77 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.351460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.382418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.383239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.383690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.384252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.385179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:36.385479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.386896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.395246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 566/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:36.400845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.472758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:36.495654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0833ecee-5bee-4b3b-a1bb-41b59279efb1 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.530968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.531589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.531861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.532128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] PATH_INFO: `/v3/users/5ffbc8b6acc747928edb425b7c6d5790` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.533223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] RBAC: Authorizing `identity:get_user(user_id=5ffbc8b6acc747928edb425b7c6d5790)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.534668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.536231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 585/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/users/5ffbc8b6acc747928edb425b7c6d5790 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:12:36.542028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.638739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b89528-70c8-4ee0-915f-12b78b477733 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.670414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.671046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.671324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.671595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] PATH_INFO: `/v3/projects/47f8e1df43d843739d22a88d09254de3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.672697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] RBAC: Authorizing `identity:get_project(project_id=47f8e1df43d843739d22a88d09254de3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.674136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c38aa3-d548-4f70-82a6-dbcd07f81c20 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.676266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 567/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:36 2022] GET /identity/v3/projects/47f8e1df43d843739d22a88d09254de3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:36.682125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:36.776566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5fb70d-71dd-4589-8149-0a18e373d508 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:36.808820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:36.809456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:36.809735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:36.810001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] PATH_INFO: `/v3/projects/47f8e1df43d843739d22a88d09254de3/users/5ffbc8b6acc747928edb425b7c6d5790/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:36.811879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=47f8e1df43d843739d22a88d09254de3, user_id=5ffbc8b6acc747928edb425b7c6d5790, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:36.813286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a68d52-2f40-4903-b004-867afe19c7bf None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:36.908853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 586/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:36 2022] PUT /identity/v3/projects/47f8e1df43d843739d22a88d09254de3/users/5ffbc8b6acc747928edb425b7c6d5790/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:38.753260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c970cf-b959-450d-8c78-92b7f61fabe1 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.753786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c970cf-b959-450d-8c78-92b7f61fabe1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.753786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c970cf-b959-450d-8c78-92b7f61fabe1 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.755135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 568/1154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:38.760323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:38.760757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:38.760851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:38.767073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:38.778981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:38.780160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:38.847032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 587/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:39.110113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c970cf-b959-450d-8c78-92b7f61fabe1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.140521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.207746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c970cf-b959-450d-8c78-92b7f61fabe1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.239793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.240409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.240687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.240960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.251485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:39.256682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:39.257778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:39.258714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 569/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:39.263973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.357842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48868995-4186-4ff5-ac32-9422fbc62708 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.385910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.390716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.391293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.391571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.391841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.392496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:39.392795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.394745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.402381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 588/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:39.407945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.495802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.504401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738673dd-3b61-4661-bde8-743cbe4df3f2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.536786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.537395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.537673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.537944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] PATH_INFO: `/v3/users/5ffbc8b6acc747928edb425b7c6d5790` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.539079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] RBAC: Authorizing `identity:get_user(user_id=5ffbc8b6acc747928edb425b7c6d5790)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.540511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.542044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 570/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/users/5ffbc8b6acc747928edb425b7c6d5790 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:39.548064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.645219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1477411-cb89-4eb8-937c-3c4256c58c51 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.677666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.678270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.678588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.678863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.735663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:39.743217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.744716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.839127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 589/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:39.845094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:39.940629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e02194-d7f4-4343-b578-b2bfd0641b66 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:39.972208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:39.972818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:39.973092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:39.973357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] PATH_INFO: `/v3/projects/47f8e1df43d843739d22a88d09254de3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:39.974472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=47f8e1df43d843739d22a88d09254de3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:39.975903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17c5e7-7650-433b-8d2a-caaa18f9c4c1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:39.977482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 571/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/projects/47f8e1df43d843739d22a88d09254de3 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:39.983471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:40.071879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:40.078117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ee815-faa9-4052-b88f-6431a1840687 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:40.110612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:40.111205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:40.111483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:40.111753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:40.112485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=47f8e1df43d843739d22a88d09254de3, user.id=5ffbc8b6acc747928edb425b7c6d5790) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:40.113442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:40.115065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f496a1-6f4e-4e9a-8b0e-48b40704a239 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:40.121046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 590/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:39 2022] GET /identity/v3/role_assignments?user.id=5ffbc8b6acc747928edb425b7c6d5790&scope.project.id=47f8e1df43d843739d22a88d09254de3&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:42.016330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8eb6-52e3-4c4f-a16a-e9e74972fd34 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.016853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8eb6-52e3-4c4f-a16a-e9e74972fd34 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.016853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed8eb6-52e3-4c4f-a16a-e9e74972fd34 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.018097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 572/1162] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:42.023222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.023632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.023761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.041325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:42.042548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:42.106014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 591/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:42.369354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed8eb6-52e3-4c4f-a16a-e9e74972fd34 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:42.397158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:42.466288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed8eb6-52e3-4c4f-a16a-e9e74972fd34 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:42.498340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:42.498953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.499229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.499494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.563857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:42.565305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:42.590436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:42.661459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 573/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:42.667417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:42.749853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:42.761453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99d9f6-80d7-458a-91fb-2b525527fbd0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:42.794146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:42.794776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.795050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.795317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.801678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:42.803129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5af11731-ded2-4c47-bc5a-cf14714a4123 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:42.813989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:42.815030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:42.816063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 592/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:12:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:12:42.821076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:42.916306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ebc41-3242-4127-81f1-a16f9f44f01b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:42.941330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:42.949026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:42.949618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:42.957360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:42.957360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:42.957360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:42.957360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:42.957360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b62b5-eb33-4dfd-8770-4a8f411d29bc None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:42.959798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 574/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 05:12:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:44.824205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827165-2343-4197-8024-ecba0cd58d57 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.824751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827165-2343-4197-8024-ecba0cd58d57 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.824751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13827165-2343-4197-8024-ecba0cd58d57 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.826084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 593/1167] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:44.831248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515827-5439-4327-84aa-2763ed2cdfc9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:44.831672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515827-5439-4327-84aa-2763ed2cdfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:44.831797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01515827-5439-4327-84aa-2763ed2cdfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:44.850171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-01515827-5439-4327-84aa-2763ed2cdfc9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:44.851478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-01515827-5439-4327-84aa-2763ed2cdfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:44.915418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 575/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:45.179825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827165-2343-4197-8024-ecba0cd58d57 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:45.278492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13827165-2343-4197-8024-ecba0cd58d57 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:45.310314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:45.310935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:45.311215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:45.311487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:45.312155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:45.313532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57af4e-85d0-4b9d-ad0d-7eda1381a794 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:45.415756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 594/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Sep 1 05:12:45 2022] POST /identity/v3/domains => generated 267 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:47.310174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa818a0-9f12-4aeb-971d-aceca631aefb None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.310739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa818a0-9f12-4aeb-971d-aceca631aefb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.310739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa818a0-9f12-4aeb-971d-aceca631aefb None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.312059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 576/1170] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:47.317706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.318117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.318253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.335920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:47.337097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:47.339040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.403159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 595/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:47.669618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa818a0-9f12-4aeb-971d-aceca631aefb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.768259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa818a0-9f12-4aeb-971d-aceca631aefb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.790163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:47.801635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.802248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.802579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.802852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] PATH_INFO: `/v3/domains/548d121c376743718170ff0909c14e52` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.812728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=548d121c376743718170ff0909c14e52)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.814182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9893d-4e0f-4e73-9339-85887d7bb47f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 577/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 1 05:12:47 2022] GET /identity/v3/domains/548d121c376743718170ff0909c14e52 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:47.821634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:47.916405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f50d6-dd52-4aab-b29f-87c60e792161 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:47.948122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:47.948708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:47.948984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:47.949257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:47.949903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:47.951349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd5a96-389a-48cb-a080-cbaa8337a5b1 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:47.960815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:48.049638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 596/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:12:47 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:49.961479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b69e38-8d59-4f73-90a6-3678f1f51a6d None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.962035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b69e38-8d59-4f73-90a6-3678f1f51a6d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.962035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b69e38-8d59-4f73-90a6-3678f1f51a6d None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.963294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 578/1174] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:49.968468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:49.968868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:49.969018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:49.986597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:49.987783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:50.051789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 597/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:50.319242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b69e38-8d59-4f73-90a6-3678f1f51a6d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.419478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b69e38-8d59-4f73-90a6-3678f1f51a6d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.451982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.452595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.452872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.453142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] PATH_INFO: `/v3/domains/548d121c376743718170ff0909c14e52` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.454243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=548d121c376743718170ff0909c14e52)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.455747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae039f2-d09e-4a4a-9c4f-735bbe69b454 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.457286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 579/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 1 05:12:50 2022] GET /identity/v3/domains/548d121c376743718170ff0909c14e52 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:50.462908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:50.557187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc383-423d-4bdd-b26b-e396adc3a8b7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:50.576117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:50.588929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:50.589521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:50.589795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:50.590062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:50.590735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:50.592149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635908-e1f3-4a5f-b911-549b5aa88ac5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:50.671540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 598/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:12:50 2022] POST /identity/v3/users => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:12:52.593303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665114d-9892-49d4-9e09-55b097535193 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.593857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665114d-9892-49d4-9e09-55b097535193 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.593857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665114d-9892-49d4-9e09-55b097535193 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.595204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 580/1178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:52.601012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:52.601401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:52.601559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:52.619189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:52.620363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:52.684496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 599/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:52.952774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665114d-9892-49d4-9e09-55b097535193 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.002871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.051684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665114d-9892-49d4-9e09-55b097535193 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.082969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.083580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.083857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.084125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.094329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:53.099565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:53.100759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:53.101870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 581/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:53.106423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.200551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89a7d4-85a1-4378-8b22-fd2efec11ce4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.208002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.233725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.234328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.234642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.234912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.235570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:53.235870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.237269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.245496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 600/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:53.251347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.347205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea6bb5-6949-43ac-8254-07a9f39553a6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.357021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.378979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.379599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.379877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.380149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] PATH_INFO: `/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.393782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] RBAC: Authorizing `identity:get_user(user_id=5cda8ee87bfa4cd4b99f76973c2d5fd1)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.395276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.396846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 582/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:53.402591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.497369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11597616-e502-40b2-8e2b-2a99613b1d57 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.529709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.530298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.530597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.530863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.562315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.594287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.595746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.688963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 601/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:53.695316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.791566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe31f-e2bb-4bf1-a42c-33ab82b3b340 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.824382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.824995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.825273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.825542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] PATH_INFO: `/v3/projects/5eef5b834fbe4a02890121fcdf687ed0` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.826670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] RBAC: Authorizing `identity:get_project(project_id=5eef5b834fbe4a02890121fcdf687ed0)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.828101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9618c-290c-4e89-b2cd-1eed6d88a598 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.829703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 583/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/projects/5eef5b834fbe4a02890121fcdf687ed0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:53.836302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:53.899701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:53.931432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553b9fd-1ff4-4144-a37b-917467190665 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:53.964191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:53.964778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:53.965052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:53.965321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:53.966030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=5eef5b834fbe4a02890121fcdf687ed0, user.id=5cda8ee87bfa4cd4b99f76973c2d5fd1) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:53.966333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:53.967770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f2466-a264-4ef5-84e1-3d44d78123cc None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:53.972964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 602/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:12:53 2022] GET /identity/v3/role_assignments?user.id=5cda8ee87bfa4cd4b99f76973c2d5fd1&scope.project.id=5eef5b834fbe4a02890121fcdf687ed0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:55.818810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aed62-162e-4817-a652-2b4d002327fa None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.819380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aed62-162e-4817-a652-2b4d002327fa None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.819380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8aed62-162e-4817-a652-2b4d002327fa None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.820660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 584/1186] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:55.825772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:55.826200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:55.826325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:55.844140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:55.845327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:55.910383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 603/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:56.173407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8aed62-162e-4817-a652-2b4d002327fa None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.207887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.270951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8aed62-162e-4817-a652-2b4d002327fa None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.302881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.303483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.303761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.304026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.313586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:56.318295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:56.319434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:12:56.320572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 585/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:56.325299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.383113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.420729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31760724-c8a6-4819-bbd7-5c36cc8686e4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.452466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.452998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.453274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.453546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.454201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:56.454528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.456392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.464043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 604/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:56.469708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.558528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.564038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80763f-c3c6-4f4b-8a6b-858086089f8b None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.596944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.597550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.597824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.598090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] PATH_INFO: `/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.599214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] RBAC: Authorizing `identity:get_user(user_id=5cda8ee87bfa4cd4b99f76973c2d5fd1)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.600644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.602175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 586/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:56.607794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.703896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15cd9d-39c1-42fb-a188-911cd4c8a107 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.725877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:56.736149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.736791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.737066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.737337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] PATH_INFO: `/v3/projects/5eef5b834fbe4a02890121fcdf687ed0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.738464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] RBAC: Authorizing `identity:get_project(project_id=5eef5b834fbe4a02890121fcdf687ed0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.739903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59b2e-a298-4d6f-883e-7b6f97482e09 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.741590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 605/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:56 2022] GET /identity/v3/projects/5eef5b834fbe4a02890121fcdf687ed0 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:56.747521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:56.842956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac7f6e-692d-457b-886c-ad396df87bbd None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:56.875302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:56.875944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:56.876224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:56.876496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] PATH_INFO: `/v3/projects/5eef5b834fbe4a02890121fcdf687ed0/users/5cda8ee87bfa4cd4b99f76973c2d5fd1/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:56.878999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] RBAC: Authorizing `identity:create_grant(project_id=5eef5b834fbe4a02890121fcdf687ed0, user_id=5cda8ee87bfa4cd4b99f76973c2d5fd1, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:56.880443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9e5e6-706e-43da-b327-c49ba64c2063 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:56.953278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 587/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:12:56 2022] PUT /identity/v3/projects/5eef5b834fbe4a02890121fcdf687ed0/users/5cda8ee87bfa4cd4b99f76973c2d5fd1/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:12:58.792115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574126d-210c-4b43-b8de-5fb86690b60d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:58.792663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574126d-210c-4b43-b8de-5fb86690b60d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:58.792663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574126d-210c-4b43-b8de-5fb86690b60d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:58.793999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 606/1193] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:12:58.799622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:58.800057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:58.800150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:58.817798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:12:58.819001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:12:58.835514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:58.885849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 588/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:12:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:12:59.149839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574126d-210c-4b43-b8de-5fb86690b60d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.246804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f574126d-210c-4b43-b8de-5fb86690b60d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.279464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.280101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.280376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.280643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.290928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:12:59.295634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:12:59.296740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:12:59.297773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 607/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:12:59.302689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.396438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ad63b-de6d-4b55-b220-ad23f16051bb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.428813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.429344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.429622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.429893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.430581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:12:59.430887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.432268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:59.439924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 589/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:59.445557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.496874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.542326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e8f947-a3da-486c-9c76-2b6cccb48008 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.574541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.575148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.575422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.575689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] PATH_INFO: `/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.576779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] RBAC: Authorizing `identity:get_user(user_id=5cda8ee87bfa4cd4b99f76973c2d5fd1)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.578187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:59.579744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 608/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/users/5cda8ee87bfa4cd4b99f76973c2d5fd1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:12:59.585580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.604267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.685423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791825-ad97-4c8d-a797-94473e8ba230 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:12:59.717184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:12:59.717780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:12:59.718056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:12:59.718325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:12:59.784340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:12:59.785790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:12:59.802265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:12:59.881215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 590/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:12:59.887476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:12:59.983858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f4be4-1400-4b2a-bda2-9b5e70baa658 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.002734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.016140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.016755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.017035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.017310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] PATH_INFO: `/v3/projects/5eef5b834fbe4a02890121fcdf687ed0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.019307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] RBAC: Authorizing `identity:get_project(project_id=5eef5b834fbe4a02890121fcdf687ed0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.020735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627afa8-5fc9-4f71-a98d-afc8c4710a41 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.022387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 609/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 05:12:59 2022] GET /identity/v3/projects/5eef5b834fbe4a02890121fcdf687ed0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:00.028609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:13:00.122886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b55c63-4c55-4aca-af00-65b3310e9a3d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:13:00.138661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:00.156120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:13:00.156712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:13:00.156992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:13:00.157263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:13:00.157986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=5eef5b834fbe4a02890121fcdf687ed0, user.id=5cda8ee87bfa4cd4b99f76973c2d5fd1) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:13:00.158899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:13:00.160416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a048572-226c-4469-bf77-a7877c683c4d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:13:00.165917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 591/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:13:00 2022] GET /identity/v3/role_assignments?user.id=5cda8ee87bfa4cd4b99f76973c2d5fd1&scope.project.id=5eef5b834fbe4a02890121fcdf687ed0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:13:10.030991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:13:10.167969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=10 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:12.814277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca97f5-00a7-44e8-ac47-89f95944ea56 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.814970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca97f5-00a7-44e8-ac47-89f95944ea56 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.814970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca97f5-00a7-44e8-ac47-89f95944ea56 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.816790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 610/1201] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:12.822116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:12.822588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:12.822748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:12.841056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:12.842275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:12.910039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 592/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:13.181926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca97f5-00a7-44e8-ac47-89f95944ea56 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.282174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca97f5-00a7-44e8-ac47-89f95944ea56 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.315416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.316042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.316316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.316587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.326642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.331646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:13.342634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:13.343913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:13.345075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:13.345075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:13.345075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:13.345075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 611/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:13.350377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.448786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93598c1-87c9-46ca-b063-f8d42d6040eb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.476753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.482456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.483059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.483334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.483600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.484234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:13.484531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.485899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ed15da-56eb-4e52-8ea6-d4f52fab4578 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.497270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 593/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:13.504080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:13.556096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:13.600735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e543c5-5e25-4565-b26f-cf544801df6f None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:13.632996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:13.633605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:13.633890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:13.634560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:13.634904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:13.636371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f847701-867d-440f-90ef-bcf15ef327d5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:13.752617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 612/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 05:14:13 2022] POST /identity/v3/users => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:15.688396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a6c-48ae-444e-a1ed-c6dd215e4a2e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:15.688924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a6c-48ae-444e-a1ed-c6dd215e4a2e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:15.688924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b9a6c-48ae-444e-a1ed-c6dd215e4a2e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:15.690251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 594/1206] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:15.695428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:15.695817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:15.695973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:15.714237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:15.715465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:15.779818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 613/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:16.047296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b9a6c-48ae-444e-a1ed-c6dd215e4a2e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.148823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b9a6c-48ae-444e-a1ed-c6dd215e4a2e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.181369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.181984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.182259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.182557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.193461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:16.198138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:16.199312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:16.200382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 595/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:16.205334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.244362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.301813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa321e4e-6588-4e48-b1f8-5b0328963851 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.333919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.334533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.334812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.335081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.335738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:16.336038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:16.337395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:16.345792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 614/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:16.352148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.400693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.448084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef100a96-4ea0-4d01-bd7f-818d0daec60d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.481438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28c816-a900-411b-af40-b7443947165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.483927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c816-a900-411b-af40-b7443947165f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.483927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c816-a900-411b-af40-b7443947165f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.483927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28c816-a900-411b-af40-b7443947165f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.493079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28c816-a900-411b-af40-b7443947165f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:16.494564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28c816-a900-411b-af40-b7443947165f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-bf28c816-a900-411b-af40-b7443947165f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:16.508609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:16.509737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:16.510711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:16.510711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:16.510711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:16.510711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 596/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:16.515560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.609578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.612568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9dc56-1393-4142-a097-9ee8223a1487 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.646714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.647324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.647604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.647866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.648505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:16.648807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:16.650194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:16.661740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 615/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:16.669553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28c816-a900-411b-af40-b7443947165f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.764461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28c816-a900-411b-af40-b7443947165f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.765957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:16.797126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.797736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.798011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.798279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.805548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:16.807027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:16.818308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:16.819472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:16.820898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 597/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:16.825244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:16.923888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c462fe8-68cc-44e6-b7e8-54c42557394c None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:16.957488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:16.958180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:16.958508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:16.958783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:16.959506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:16.959811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:16.961272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:16.977754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 616/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:16.983678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.085730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bc1b4-49d5-4516-8486-a850a82bc9e2 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.118222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.118882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.119156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.119422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.149026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.185673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.187089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.282541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 598/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:17.288296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.306998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.385623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6221cb4-6a0e-415f-9370-45891f5378d5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.419060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.419705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.419979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.420247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.430838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.432263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:17.446662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:17.447798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:17.448730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:17.448730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:17.448730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:17.448730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 617/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:17.453607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.514309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.551202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775eac6-2a3f-4aec-8fc4-9be8242d2778 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.582973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.583594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.583867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.584133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.584763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.585060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.586448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.597715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 599/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:17.603309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.673568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.703773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52e16e-0c3a-4ebb-a29f-91cb2dc72829 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.736639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.737773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.738049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.738313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.744846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.746281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:17.758186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:17.759294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:17.760407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 618/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:17.765224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:17.862685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c05134-b861-4cb2-8cc7-7e8068ce7fb8 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:17.885403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:17.896721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:17.897321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:17.900463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:17.900463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:17.900463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:17.900463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:17.900463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acad8f-1fa7-4df8-8310-f66df27b9ed6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:17.911747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 600/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:17.917709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:18.015898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6966c-925a-4cea-b154-a1ff591688fa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:18.043345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:18.048917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:18.049546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:18.049807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:18.050074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:18.050812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=0c57f8b0816a4832b24043daea93e113) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:18.051118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:18.052530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04343cb2-82f5-4451-88ea-6897b5272b1b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:18.057831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 619/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:17 2022] GET /identity/v3/role_assignments?user.id=0c57f8b0816a4832b24043daea93e113&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:19.951415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9c755-7424-48fb-9638-4d77aca6e2c8 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:19.952018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9c755-7424-48fb-9638-4d77aca6e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:19.952018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9c755-7424-48fb-9638-4d77aca6e2c8 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:19.953192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 601/1220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:19.958935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:19.959378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:19.959470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:19.977881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:19.979130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:20.042888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 620/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:20.315701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9c755-7424-48fb-9638-4d77aca6e2c8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.417101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9c755-7424-48fb-9638-4d77aca6e2c8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.451355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.451973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.452245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.452510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.462867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:20.467834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:20.470465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:20.471607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 602/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:20.474975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.579409 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b0b1b-3447-49cb-8707-5f7c10775c37 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.602341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:20.613270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.613870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.614149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.614440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.615105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:20.615406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:20.616904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:20.624548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 621/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:20.630605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.675152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:20.728629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f64d-cc55-4d5f-bc69-44e5852df184 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.761181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.761805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.762082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.762350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.772489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:20.773959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:20.789205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:20.790386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:20.791583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:20.791583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:20.791583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:20.791583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 603/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:20.796113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:20.895166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29451eb-65d9-4a88-a398-b70a5a9f70f2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:20.928350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:20.928945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:20.929219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:20.929485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:20.930114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:20.930437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:20.931836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:20.943915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 622/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:20.949648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.041183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:21.047757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01702cc-c5ab-4e0d-af68-4b35c836e4f1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.082425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.083057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.083337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.083606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.090783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.092230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:21.103909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:21.105075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:21.106311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 604/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:21.110932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.157716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:21.208570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29104e-79b4-4ae7-ac0d-6d2b7cefa67b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.253062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.253915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.254212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.254506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.255228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:21.255524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.256943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.274370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 623/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:21.279854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.382316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808d6e7-9eca-480d-919d-a9bcde3f8205 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.415299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.415865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.416141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.416407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.426528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.427977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:21.442333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:21.443505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:21.444472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:21.444472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:21.444472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:21.444472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 605/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:21.449430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.547133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b37c7-675f-4d9c-808f-6be02d73fffa None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.550171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:21.585399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.589109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.599906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 624/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:21.605623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.615339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:21.703343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30b5b1-eb82-4cb0-82a2-80de41ced321 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.735944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.736573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.737350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.737620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.744428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.745876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:21.757588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:21.759788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:21.760961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 606/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:21.764704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.861605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb823e-0bde-49a5-9284-a1616c54f583 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:21.896051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:21.896648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:21.896922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:21.897198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:21.897876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:21.898178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:21.899736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac0764-4994-4a1a-a972-07e10c9e298c None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:21.912245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 625/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:21.918304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:21.982016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:22.015772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5a63b-29e3-40a1-82ea-9ec0541a9bed None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:22.047651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:22.048289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:22.048563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:22.048832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/0c57f8b0816a4832b24043daea93e113/roles/266eb329bccf4539ab5413c561ff8dc9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:22.064095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=0c57f8b0816a4832b24043daea93e113, role_id=266eb329bccf4539ab5413c561ff8dc9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:22.066205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a970e28-d3e2-4798-8f4d-5991d4e65fdd None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:22.189563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 607/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:21 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/0c57f8b0816a4832b24043daea93e113/roles/266eb329bccf4539ab5413c561ff8dc9 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:24.050100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e66ce-ed55-4899-8b92-42d7b57ef3e8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.050686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e66ce-ed55-4899-8b92-42d7b57ef3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.050686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e66ce-ed55-4899-8b92-42d7b57ef3e8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.052039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 626/1233] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:24.057214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.057618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.057754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.076016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:24.082538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:24.142179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 608/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:24.408185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e66ce-ed55-4899-8b92-42d7b57ef3e8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:24.432613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:24.506326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e66ce-ed55-4899-8b92-42d7b57ef3e8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:24.540784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:24.541407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.541684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.541955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.552297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:24.557085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:24.558222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:24.559380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 627/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 05:14:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:24.564396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:24.609214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:24.660826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786f3b3-4a3d-4fb6-8497-569de88f2f85 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:24.695429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:24.696592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.696873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.697140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.697802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:24.698103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:24.699655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:24.707638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 609/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 05:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:24.713446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:24.800712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:24.816056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd34a51-1a89-4fc4-abde-b335d3c3c081 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:24.848255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:24.848879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:24.849262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:24.849531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:24.859606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:24.861041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:24.875616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:24.876801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:24.878221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:24.878221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:24.878221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:24.878221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 628/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:24.883002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:24.980553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df7302-b835-46d2-8d67-25fdba162c08 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:24.989769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:25.013710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.014329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.014658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.014931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.015589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:25.015912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.017300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.028619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 610/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:14:25.034928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:25.131673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c157147-1fb4-4305-961d-1ce6b31475fe None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:25.163867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.164492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.164766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.165040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.167705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:25.172436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.174539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:25.185153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:25.190282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:25.191395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 629/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:25.192462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:25.289752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e350d88-1f31-481a-a292-eb6eda5f9d4f None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:25.322102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.322728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.323004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.323271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.323994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:25.324295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.325729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.346587 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 611/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:25.352441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:25.453138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4c9c-e060-446a-b170-c05495aa3fa2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:25.487583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.488183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.488462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.488738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.548244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:25.554647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.563831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.654720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 630/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:25.661262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:25.688291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:25.757560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a87121-5064-46a9-9936-769c13f66ced None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:25.791380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135626-357b-4c62-884d-e176056ced01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.792003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135626-357b-4c62-884d-e176056ced01 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.792282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135626-357b-4c62-884d-e176056ced01 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.792552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35135626-357b-4c62-884d-e176056ced01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.803387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35135626-357b-4c62-884d-e176056ced01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.804816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35135626-357b-4c62-884d-e176056ced01 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-35135626-357b-4c62-884d-e176056ced01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:25.819396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:25.820553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:25.821693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:25.821693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:25.821693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:25.821693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 612/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:25.826398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:25.918351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:25.924728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a81279-9e17-4673-8736-feebb0a70788 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:25.957865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:25.958495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:25.958771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:25.959056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:25.959672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:25.959973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:25.961351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:25.972967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 631/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:25.978846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135626-357b-4c62-884d-e176056ced01 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.056215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:26.076960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35135626-357b-4c62-884d-e176056ced01 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.109860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.110502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.110784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.111055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.118019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.119499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:26.131281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:26.132337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:26.133447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 613/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:26.140227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.237928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224480e-9519-4578-9448-b30cf351ea99 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.270843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.271438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.271709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.271975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.272648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:26.272945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.274496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d745315-f128-4e25-9bed-59856dc3e5ca None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.285859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 632/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:26.292063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:26.389163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391ed95-603d-40ac-b46b-bd8d5882fef7 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:26.421077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:26.421663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:26.421937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:26.422201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:26.422943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] RBAC: Adding query filter params (role.id=266eb329bccf4539ab5413c561ff8dc9, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=0c57f8b0816a4832b24043daea93e113) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:26.423245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:26.425331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d26fc-bd48-49c5-8404-01755310451f None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:26.430849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 614/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:26 2022] GET /identity/v3/role_assignments?user.id=0c57f8b0816a4832b24043daea93e113&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=266eb329bccf4539ab5413c561ff8dc9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:28.400908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97da5-bd09-4bca-9188-ba67fad56370 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:28.401470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97da5-bd09-4bca-9188-ba67fad56370 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:28.401470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c97da5-bd09-4bca-9188-ba67fad56370 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:28.402850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 633/1247] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:28.407884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:28.408865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:28.409058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:28.415323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:28.427673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:28.430475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:28.494586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 615/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:28.766035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97da5-bd09-4bca-9188-ba67fad56370 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:28.784604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:28.865226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c97da5-bd09-4bca-9188-ba67fad56370 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:28.899504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:28.900116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:28.900395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:28.900668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:28.910538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:28.915368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:28.916593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:28.917527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 634/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:28.922591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.019134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676089f3-30c0-476f-ab0e-73f6ab1d8469 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.047401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.052932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.053537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.053840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.054118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.054817 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:29.055126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.056577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.065127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 616/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:29.070958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.147052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.169415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eb540-1083-4d69-a0bb-0e9fee4a7a21 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.202991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.203620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.203899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.204172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.214366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.215842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:29.230173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:29.231338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:29.232457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:29.232457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:29.232457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:29.232457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 635/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:29.237724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.336379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3ac3b-4ba1-44ba-b8f8-fefba71cf8a3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.368801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.369390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.369662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.369931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.370586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:29.370916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.372276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.383760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 617/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:29.389538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.486958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d9c5d-8057-4c4f-8d72-c98e0e60d388 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.512768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.520327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.520950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.521225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.521493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.528627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.530471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:29.541276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:29.542462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:29.543655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 636/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:29.548305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.603743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.646688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d2e0f-915f-46b4-a5c7-2a81258cdb1d None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.680759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.681361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.681639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.681920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.682669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:29.682972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.684413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.700784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 618/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:29.707096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:29.803815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176bb9f-3653-4a7b-8398-71ae58bf24ce None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:29.835975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:29.836574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:29.836849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:29.837119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:29.890305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:29.902367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:29.903864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:29.997989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 637/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 05:14:30.004007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.098510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcac22f-ebd8-4042-8a95-83e5de6ce8a1 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.110996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:30.131669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.132295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.132568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.132835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.142436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.144447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:30.158328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:30.159477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:30.160404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:30.160404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:30.160404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:30.160404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 619/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:30.165235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.250642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:30.262808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9b3b8-a303-4cf3-8da7-6acfda73ed39 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.295349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.295949 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.296227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.296500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.297138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:30.297442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.298836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.310169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 638/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:30.315701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.412538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e60b-70b4-4402-9fe9-dabca6cb4079 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.445243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.445859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.446136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.446430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.452811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.454264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:30.465186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:30.466313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:30.467456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 620/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:30.472133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.569896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8ef5c-2b1f-42ef-b380-53c10a1fd35b None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.601843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.602464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.602744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.603013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.603695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:30.603999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.605413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb0ed1-c216-42bc-a010-03db4610acf2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.618042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 639/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:30.624149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:30.650648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:30.721424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96e869-d5d0-4af6-8824-ae72751faf79 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:30.755294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:30.755890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:30.756165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:30.756432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:30.757143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=0c57f8b0816a4832b24043daea93e113) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:30.757445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:30.758925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04301e-9148-43ae-8dcf-cffcf59d8cb8 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:30.764045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 621/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:30 2022] GET /identity/v3/role_assignments?user.id=0c57f8b0816a4832b24043daea93e113&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:32.608783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760747c-7c5d-4bcf-9366-f128c670cf68 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.609342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760747c-7c5d-4bcf-9366-f128c670cf68 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.609342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760747c-7c5d-4bcf-9366-f128c670cf68 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.610837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 640/1261] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:32.616792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:32.617240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:32.617334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:32.637474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:32.637474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:32.704023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 622/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:32.990959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760747c-7c5d-4bcf-9366-f128c670cf68 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.001612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.091286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760747c-7c5d-4bcf-9366-f128c670cf68 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.123847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.124462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.124737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.125001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.135538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:33.140291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:33.141423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:33.142521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 641/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:33.149122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.166788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.242303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34d43-c720-48dd-a50e-c9ad1a0dc77e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.278700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.279296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.279571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.279836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.280487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:33.280784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:33.282240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:33.290154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 623/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:33.295976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.363637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.395776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32228bc-64f7-4cc8-8c72-4b2b0ae7fc08 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.428265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.428895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.429159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.429429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.439053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:33.441068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:33.455476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:33.456704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:33.457707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:33.457707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:33.457707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:33.457707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 642/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:33.462617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.540798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.562008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c567b67-aa8e-4598-8bbf-f6d84576ad24 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.594738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.595334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.595608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.595873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.596526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:33.596825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:33.598204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:33.610257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 624/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:33.616487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.714892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e854f-4bb3-43e8-a5f9-6199679ed7d2 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.734093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.748392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.749012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.749287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.749555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.757634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:33.759083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:33.769721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:33.770934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:33.772302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 643/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:33.776927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:33.882009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0ca8e-4a3a-4107-91b3-27c65a080d86 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:33.915599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:33.916219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:33.916504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:33.916776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:33.917517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:33.917822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:33.919391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:33.929300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:33.939698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 625/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:33.945453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:34.042963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee78e6-8830-41ad-9041-9a3d0efb4b6e None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:34.075534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:34.076152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.076426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.076728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.087455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:34.088899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:34.103099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:34.111530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:34.113208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:34.113208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:34.113208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:34.113208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 644/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 05:14:34.113497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:34.206733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ba683-4026-48c4-93c3-77d3bcb7840e None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:34.239658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:34.240267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.240534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.240802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.241437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:34.241738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:34.243135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:34.254322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 626/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:34.260244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:34.291842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:34.358641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8a2d4-68b3-4baa-93ec-621bfa90fab9 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:34.390859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:34.391511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.391788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.392065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.398932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:34.400439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:34.412896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:34.413999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:34.415163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 645/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:34.420073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:34.477669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:34.517185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e712a8f-a353-459b-86b8-e00475c4d208 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:34.551266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:34.551862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.552136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.552404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.553087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:34.553386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:34.554865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c448bc9-16a0-49a3-b393-ab1551fad53b None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:34.567123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 627/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:34.573160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:34.661173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:34.669688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8116bf-815a-4513-830d-610cbdf6c677 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:34.702034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:34.702710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:34.702988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:34.703259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] PATH_INFO: `/v3/projects/27a422cf07fd4997b80019272e0addc0/users/0c57f8b0816a4832b24043daea93e113/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:34.705159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a422cf07fd4997b80019272e0addc0, user_id=0c57f8b0816a4832b24043daea93e113, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:34.707175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cbfe4-f83e-4d5b-8e38-e26b850ca6b3 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:34.800157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 646/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 05:14:34 2022] PUT /identity/v3/projects/27a422cf07fd4997b80019272e0addc0/users/0c57f8b0816a4832b24043daea93e113/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:14:36.715927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c50f-59d5-4d3a-94df-78d649580c5c None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:36.716447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c50f-59d5-4d3a-94df-78d649580c5c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:36.716447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c50f-59d5-4d3a-94df-78d649580c5c None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:36.717787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 628/1274] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:36.722890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:36.723355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:36.723449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:36.741749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:36.742955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:36.808385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 647/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:37.087327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c50f-59d5-4d3a-94df-78d649580c5c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.188577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c50f-59d5-4d3a-94df-78d649580c5c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:37.206754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:37.223782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:37.224406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.224679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.224944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.235632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 05:14:37.240607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:37.241774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:37.242775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 629/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:37.247908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.327877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:37.345588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7390b-f68a-4009-a5a1-cb85027bc4b1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:37.378465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:37.379079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.379357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.379632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.380303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:37.380610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:37.382012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:37.390125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 648/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:37.396460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.496791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e52296-5623-4f86-88f3-17a94ae05ec5 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:37.529432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:37.530048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.530322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.530654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.540868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:37.542317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:37.556177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:37.557287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:37.558225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:37.558225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:37.558225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:37.558225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 630/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:37.563374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.664321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f65fb-f642-4d46-9ce1-30133a1e34d5 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:37.697539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:37.698134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.698434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.698715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.699357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:37.699660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:37.701043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:37.713049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 649/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:37.718822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.748425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:37.814469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d2cc1-5f7b-4f2a-9196-6c840e2025d6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:37.848039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:37.848651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:37.848925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:37.849187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:37.856372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:37.857809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 05:14:37.868607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:37.872790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:37.873808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 631/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:37.875717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:37.880589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:37.972838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c11ec-212e-4ede-813e-ccbc7cbed343 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.005842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.006456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.006733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.007000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.007711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:38.008011 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.009420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.025573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 650/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 05:14:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:38.031524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.122586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.132969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d3664-7d1d-4c12-bdbf-88c8614be5ee None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.165176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.165775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.166095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.166445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.233548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.235017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.322608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.332407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 632/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 05:14:38.338469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.399492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.436672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4580fe-6e64-4ce5-a4d9-dce8eb6e9759 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.469341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.469963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.470243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.470558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.480729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.482162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 05:14:38.497394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:38.498366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 05:14:38.499451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 05:14:38.499451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 05:14:38.499451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:38.499806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 651/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:38.504682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.599729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e79af-ad4c-42e1-bbca-4c923c74e4cf None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.633457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.634059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.634783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.634783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.635260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:38.635562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.636937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.648688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 633/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:38.654367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.750963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bbbe2d-f200-43b1-abc7-32c6f3caab20 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.765509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.786908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.787534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.787811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.788081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.794664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.796119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 05:14:38.807073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:38.811760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: ERROR keystone.server.flask.application Sep 01 05:14:38.812720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 652/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:38.814275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:38.863187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:38.911738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdafbc-de66-4960-ac52-2348f79fc307 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:38.943473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:38.944064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:38.944339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:38.944605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:38.945315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:38.945602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:38.947065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff509de-d000-4b36-9d06-357148521902 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:38.959742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 634/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:38.965955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:39.063778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d5b1-c998-4e9b-bc79-497ec446476d None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:39.096884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:39.097584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:39.097861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:39.098128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:39.098875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] RBAC: Adding query filter params (role.id=875fac1137784b2fb2938ba0c2c98083, scope.project.id=27a422cf07fd4997b80019272e0addc0, user.id=0c57f8b0816a4832b24043daea93e113) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:39.099179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:39.100608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e01204-2889-4ea3-bdf9-1d44701290a4 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:39.106192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 653/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 05:14:38 2022] GET /identity/v3/role_assignments?user.id=0c57f8b0816a4832b24043daea93e113&scope.project.id=27a422cf07fd4997b80019272e0addc0&role.id=875fac1137784b2fb2938ba0c2c98083 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:40.996250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8566-8467-46d9-b2b1-cf8b6857b52b None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:40.996828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8566-8467-46d9-b2b1-cf8b6857b52b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:40.996828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8566-8467-46d9-b2b1-cf8b6857b52b None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:40.998031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 635/1288] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:41.004072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.004522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.004614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.022296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:41.023506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:41.052140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:41.087675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 654/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 05:14:41.352144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8566-8467-46d9-b2b1-cf8b6857b52b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.452088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8566-8467-46d9-b2b1-cf8b6857b52b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.462539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:41.485458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.486085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.486362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.486667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.487251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.489244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:41.497781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:41.499190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:41.500391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 636/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:41.505174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.608765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b09291-4eb5-440d-97e8-746a9f5493b3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.645198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.645823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.646097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.646391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.647038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:41.647336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.648700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6c446-e284-4850-9c89-9e7d10529ddd None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.653319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 655/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:41.659081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:41.757763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ad5e6-668e-4d71-a0a7-c7dede0ce053 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:41.790377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:41.790997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:41.791271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:41.791539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] PATH_INFO: `/v3/services` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:41.792161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:41.792456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:41.793822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4380a95-ff53-450c-9823-d1253a2e49b1 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:41.798056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 637/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 05:14:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:43.694787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f912dd-f6ec-43b3-b72b-ee441a3a5b71 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:43.695335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f912dd-f6ec-43b3-b72b-ee441a3a5b71 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:43.695441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f912dd-f6ec-43b3-b72b-ee441a3a5b71 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:43.696719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 656/1293] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:43.701983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149ed1-33b2-4239-89ab-5bae2b547ce9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:43.702394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149ed1-33b2-4239-89ab-5bae2b547ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:43.703897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b149ed1-33b2-4239-89ab-5bae2b547ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:43.721055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4b149ed1-33b2-4239-89ab-5bae2b547ce9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:43.722272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b149ed1-33b2-4239-89ab-5bae2b547ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:43.750973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:43.788015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 638/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:44.057687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f912dd-f6ec-43b3-b72b-ee441a3a5b71 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:44.088369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:44.156918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f912dd-f6ec-43b3-b72b-ee441a3a5b71 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:44.189813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:44.190427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:44.191258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:44.191529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:44.192166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:44.193521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20f9b0-3722-4509-825a-39217bc80b7b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:44.253041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 657/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 05:14:44 2022] POST /identity/v3/services => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:46.180706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584bf8-4b21-4094-b4a3-431de752f510 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:46.181291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584bf8-4b21-4094-b4a3-431de752f510 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:46.181291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584bf8-4b21-4094-b4a3-431de752f510 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:46.182641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 639/1296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:46.187893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:46.188394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:46.188492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:46.206562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:46.207742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:46.279021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 658/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:46.548035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584bf8-4b21-4094-b4a3-431de752f510 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:46.647903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584bf8-4b21-4094-b4a3-431de752f510 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:46.680369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:46.681122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:46.681401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:46.681667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:46.682242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:46.683652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:46.692875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:46.698646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:46.699802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 640/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:14:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:46.700805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:46.718443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:46.796903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc050e9-fb58-4ace-94de-17b3139be524 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:46.830373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:46.831574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:46.831854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:46.832123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:46.832750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:46.833053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:46.834440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:46.839599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 659/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 05:14:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:46.845560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:46.876267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:46.943068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001d314-d471-4ef8-9cc3-da2b495a15d6 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:46.977539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:46.978137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:46.978438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:46.978719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:46.979420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] RBAC: Adding query filter params (interface=public, service_id=0fa91e83ba134fd695dd515a14f2c237, region_id=RegionOne) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:46.979732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:46.981118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ae2a7-b720-48e5-a792-a5e555bc1661 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:46.986083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 641/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 05:14:46 2022] GET /identity/v3/endpoints?service_id=0fa91e83ba134fd695dd515a14f2c237&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:46.992466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:47.073114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:47.090533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0f727-0d78-4e8c-abb4-5a1c6d434c81 None admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:47.123783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:47.124386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:47.124660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:47.124930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:47.125825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:47.127563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500be74b-9050-499a-b43a-32b352a2b682 None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:47.135231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 660/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 05:14:46 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:14:49.017100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a748664-5215-4b6a-9614-648830fa8c75 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:49.017639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a748664-5215-4b6a-9614-648830fa8c75 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:49.017639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a748664-5215-4b6a-9614-648830fa8c75 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:49.018990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 642/1302] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:14:49.023999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:49.024470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:49.024566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:49.042687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:14:49.043851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:14:49.108309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 661/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:14:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 05:14:49.376056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a748664-5215-4b6a-9614-648830fa8c75 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:49.469796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:49.477042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a748664-5215-4b6a-9614-648830fa8c75 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:49.509383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:49.510006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:49.510303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:49.510592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:49.511169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:49.512541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 05:14:49.521765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 05:14:49.522986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application with Lock( Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self._enter() Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application return self.creator() Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application created_value = creator( Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: ERROR keystone.server.flask.application Sep 01 05:14:49.524225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 643/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 05:14:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 05:14:49.528871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:49.605800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:49.627300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab23e29-6ce4-407a-b6ef-f75f23723f84 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:49.660728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:49.661356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:49.661599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:49.661865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] PATH_INFO: `/v3/services` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:49.662511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:14:49.662809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:49.664167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6beaf-5999-4c2d-b6d5-9381336c574b None admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:49.669085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 662/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 05:14:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:14:49.674890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:14:49.771760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e3c97-fcd4-40ed-8c89-ad0e36cd4cf3 None admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:14:49.803182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:14:49.803766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:14:49.804040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:14:49.804305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:14:49.804963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:14:49.806328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88eeb7-15cf-4a89-b754-07fa933f73e7 None admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:14:49.900708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 644/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:14:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:14:59.666098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=30 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:14:59.896119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:09.679667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa7d85-e306-40c3-bf15-ab1a217e8e47 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:09.680185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa7d85-e306-40c3-bf15-ab1a217e8e47 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:09.680282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa7d85-e306-40c3-bf15-ab1a217e8e47 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:09.681722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 663/1307] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:09.687773 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d121e4-cfd2-4b82-b73f-875e306c516e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:09.688205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d121e4-cfd2-4b82-b73f-875e306c516e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:09.688378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d121e4-cfd2-4b82-b73f-875e306c516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:09.769464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c4d121e4-cfd2-4b82-b73f-875e306c516e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:09.770735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c4d121e4-cfd2-4b82-b73f-875e306c516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:09.894570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 645/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:10.193320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03788c7-67ef-488b-8483-532488176474 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:10.193795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03788c7-67ef-488b-8483-532488176474 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:10.194145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03788c7-67ef-488b-8483-532488176474 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:10.195851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 664/1309] 10.222.0.16 () {54 vars in 965 bytes} [Thu Sep 1 05:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:10.202145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a7285-8228-4f3b-8ee5-bc3e15b6f823 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:10.202532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a7285-8228-4f3b-8ee5-bc3e15b6f823 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:10.202906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a7285-8228-4f3b-8ee5-bc3e15b6f823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:10.281451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f59a7285-8228-4f3b-8ee5-bc3e15b6f823 None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:10.282749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f59a7285-8228-4f3b-8ee5-bc3e15b6f823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:10.310882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:10.371836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 646/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Sep 1 05:21:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:10.381095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03788c7-67ef-488b-8483-532488176474 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:10.382956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b03788c7-67ef-488b-8483-532488176474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:10.512427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03788c7-67ef-488b-8483-532488176474 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:10.591478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:10.592045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:10.592325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:10.592595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:10.594214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:10.658438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:10.714474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:10.715841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcdd46-10b0-4e74-a3c9-f17e48209758 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:10.741078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 665/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:12.938874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:12.939399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:12.939399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:12.940876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 647/1312] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:12.945750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac72b-2a08-43d9-aafd-559c4fe2daf9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:12.946179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac72b-2a08-43d9-aafd-559c4fe2daf9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:12.946284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac72b-2a08-43d9-aafd-559c4fe2daf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:13.041806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-056ac72b-2a08-43d9-aafd-559c4fe2daf9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:13.043069 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-056ac72b-2a08-43d9-aafd-559c4fe2daf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:13.150688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 666/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:13.435227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:13.436749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:13.540167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65245dbf-27b8-4b72-ad98-648771e9f198 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:13.610055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:13.612914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:13.613524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:13.613804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:13.614073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:13.615724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:13.729244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:13.730737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b060aff-eb87-46b3-917e-a18bfc16129d service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:13.749325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 648/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:16.166038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:16.166639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:16.166639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:16.167959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 667/1315] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:16.174087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928d7e-e7b2-4608-9205-d747971232e5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:16.174554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928d7e-e7b2-4608-9205-d747971232e5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:16.174776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e928d7e-e7b2-4608-9205-d747971232e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:16.238594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-1e928d7e-e7b2-4608-9205-d747971232e5 None None] MFA Rules not processed for user `232b01a4248c49dda4c1e2c56d05fce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:16.239835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1e928d7e-e7b2-4608-9205-d747971232e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:16.292275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:16.340313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 649/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:16.627196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:16.628737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:16.639132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:16.737941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6665b-c22a-4f7c-a87b-9c2c14bef8aa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:16.809284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:16.809884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:16.810160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:16.810978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:16.812561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:16.872281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:16.917578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:16.919043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221eef0-a20c-4bdd-a604-bdbece76c4c3 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:16.938850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 668/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:19.698030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:19.698596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:19.698596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:19.699919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 650/1318] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:19.704991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd206-6c1a-4abb-98da-5548baf0e263 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:19.705420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd206-6c1a-4abb-98da-5548baf0e263 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:19.705515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfd206-6c1a-4abb-98da-5548baf0e263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:19.769594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b7cfd206-6c1a-4abb-98da-5548baf0e263 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:19.770823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b7cfd206-6c1a-4abb-98da-5548baf0e263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:19.864324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 669/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:20.146665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:20.148216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:20.249475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c50e-7862-446f-9c1d-d3494a767f14 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:20.319870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:20.320461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:20.320734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:20.321001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:20.322589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:20.330230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:20.431686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:20.433080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859958bb-0fdd-47b2-8a4d-8df4b8fa9123 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:20.451386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 651/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:23.248664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60daa-24eb-4d4a-9c50-5307ecf3cae5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:23.249204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60daa-24eb-4d4a-9c50-5307ecf3cae5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:23.249204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f60daa-24eb-4d4a-9c50-5307ecf3cae5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:23.250586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 670/1321] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:23.255917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184daf2-79eb-485a-80a4-b4edce918605 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:23.256363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184daf2-79eb-485a-80a4-b4edce918605 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:23.256478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184daf2-79eb-485a-80a4-b4edce918605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:23.324025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8184daf2-79eb-485a-80a4-b4edce918605 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:23.325230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8184daf2-79eb-485a-80a4-b4edce918605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:23.385727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:23.425063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 652/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:23.719729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:23.720015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:23.720314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:23.721808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 671/1323] 10.222.0.16 () {54 vars in 965 bytes} [Thu Sep 1 05:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:23.727690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8606c04-662c-486f-8626-835c30d46757 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:23.727977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8606c04-662c-486f-8626-835c30d46757 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:23.728247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8606c04-662c-486f-8626-835c30d46757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:23.804530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b8606c04-662c-486f-8626-835c30d46757 None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:23.805722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b8606c04-662c-486f-8626-835c30d46757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:23.879369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 653/1324] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Sep 1 05:21:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:23.888743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:23.890342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:23.909888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:23.992322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b624ed-bed9-497c-8493-0c2aba9f3df9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:24.063677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:24.064299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:24.064576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:24.064847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:24.066548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:24.138899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:24.176748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:24.178179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987a74a-d2d4-4872-b26a-17522b53aee8 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:24.195758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 672/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:26.389738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:26.390254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:26.390254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:26.391662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 654/1326] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:26.396843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1fb3-dd60-43c0-b5d8-fbd649f21483 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:26.397231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1fb3-dd60-43c0-b5d8-fbd649f21483 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:26.397422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1fb3-dd60-43c0-b5d8-fbd649f21483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:26.471500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5fec1fb3-dd60-43c0-b5d8-fbd649f21483 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:26.472714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5fec1fb3-dd60-43c0-b5d8-fbd649f21483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:26.569764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 673/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:26.850811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:26.852352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:26.913342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:26.955232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaa6c7-5c45-4a13-910f-c6bdfd225276 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:27.032722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:27.033395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:27.033716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:27.033988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:27.035632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:27.144722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:27.146217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d8d85-7418-4974-8712-1cb9a7f2e195 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:27.150578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:27.165180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 655/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:29.133236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:29.133236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:29.133236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:29.133236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 674/1329] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:29.136107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135d8-467c-47b3-837e-cda519338ab5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:29.136512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135d8-467c-47b3-837e-cda519338ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:29.136663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01135d8-467c-47b3-837e-cda519338ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:29.202858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e01135d8-467c-47b3-837e-cda519338ab5 None None] MFA Rules not processed for user `232b01a4248c49dda4c1e2c56d05fce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:29.204064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e01135d8-467c-47b3-837e-cda519338ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:29.321712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 656/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:21:29.615305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:29.616848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:29.649105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:29.720636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89961b0-98b4-4ab5-a80f-bf56de5a370d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:29.792690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:29.793297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:29.793573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:29.793840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:29.796037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:29.875104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:29.908338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:29.910072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd2d82-48e2-4a53-9f6b-7908fdbb9302 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:29.929029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 675/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:32.244496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:32.245181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:32.245181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:32.246464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 657/1332] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:32.251684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3dd30-7b04-4318-bad0-f4e1967c14f1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:32.252130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3dd30-7b04-4318-bad0-f4e1967c14f1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:32.252235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3dd30-7b04-4318-bad0-f4e1967c14f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:32.318214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c8f3dd30-7b04-4318-bad0-f4e1967c14f1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:32.319495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c8f3dd30-7b04-4318-bad0-f4e1967c14f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:32.416198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 676/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:32.699899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:32.702018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:32.770080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:32.804099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86e72b-49d8-40f4-ab8f-a5f0cdb5d60b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:32.874623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:32.875214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:32.875488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:32.875751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:32.877313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:32.984130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:32.985745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b497bb-7c5a-430c-86ad-486328b9dfa6 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:32.995392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:33.003260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 658/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:34.928087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:34.928610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:34.928610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:34.929907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 677/1335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:34.935973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351bc-f8f2-4e26-a87c-2ce243c0092e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:34.936424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351bc-f8f2-4e26-a87c-2ce243c0092e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:34.936529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351bc-f8f2-4e26-a87c-2ce243c0092e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:35.000789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9f0351bc-f8f2-4e26-a87c-2ce243c0092e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:35.001979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9f0351bc-f8f2-4e26-a87c-2ce243c0092e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:35.094710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 659/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:35.377965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:35.379565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:35.418346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:35.480372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd5f7f-13e2-4b72-9158-8501cfe24cc0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:35.552152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:35.552755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:35.553036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:35.553309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:35.554947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:35.636159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:35.661710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:35.663171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce20c3-641f-4ea8-9a78-41f61cebcf90 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:35.681644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 678/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:38.118937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:38.119479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:38.119479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:38.120816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 660/1338] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:38.126006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf922-5a38-476e-b8e2-f010999a590f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:38.127773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf922-5a38-476e-b8e2-f010999a590f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:38.127773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdf922-5a38-476e-b8e2-f010999a590f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:38.191631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f3fdf922-5a38-476e-b8e2-f010999a590f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:38.192840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f3fdf922-5a38-476e-b8e2-f010999a590f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:38.286246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 679/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:38.566429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:38.567978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:38.642189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:38.668594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd923f95-3405-4cde-bf61-ddb090ea209e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:38.740093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:38.740686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:38.740957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:38.741219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:38.742819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:38.853966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:38.853966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16fce-f958-4850-9b8a-8baa51a23980 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:38.872359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 661/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:42.943561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:42.944100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:42.944219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:42.945737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 680/1341] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:42.951574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7216f-f406-4307-900b-ca53642c9336 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:42.952007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7216f-f406-4307-900b-ca53642c9336 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:42.952103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7216f-f406-4307-900b-ca53642c9336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:42.957463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:43.021274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-38b7216f-f406-4307-900b-ca53642c9336 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:43.022518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-38b7216f-f406-4307-900b-ca53642c9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:43.118196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 662/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:43.404263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:43.405857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:43.453915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:43.510813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953f2b1-84e7-4977-9ae9-18722648291d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:43.582863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:43.583472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:43.583745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:43.584010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:43.585637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:43.679406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:43.695953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:43.697386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73deaaa-0f74-4fea-bb20-98a65e3dfe17 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:43.716204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 681/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:46.567531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:46.568095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:46.568095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:46.569455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 663/1344] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:21:46.574736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f23bb-ae7b-47c0-9440-3e885d58e2d8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:46.575240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f23bb-ae7b-47c0-9440-3e885d58e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:46.575339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f23bb-ae7b-47c0-9440-3e885d58e2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:46.642662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dd8f23bb-ae7b-47c0-9440-3e885d58e2d8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:46.643869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dd8f23bb-ae7b-47c0-9440-3e885d58e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:46.738894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 682/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:21:47.019524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:47.021072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:47.110773 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:47.128591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc443f-8f5b-4503-8509-89bd04688fd6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:47.203534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:47.204251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:47.204525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:47.204791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:47.206369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:47.319196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:47.320661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9cda0-0e51-43db-863c-22bd073b4bd9 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:47.339270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 664/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:51.408452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:51.409065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:51.409748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:51.416901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 683/1347] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:21:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:51.419959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81dd4-df1e-4f1e-8419-6f0b8eb7c48a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:51.432876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81dd4-df1e-4f1e-8419-6f0b8eb7c48a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:51.432876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81dd4-df1e-4f1e-8419-6f0b8eb7c48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:51.456613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:51.516650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-19c81dd4-df1e-4f1e-8419-6f0b8eb7c48a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:51.517867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-19c81dd4-df1e-4f1e-8419-6f0b8eb7c48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:51.618464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 665/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:21:51.899965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:21:51.901534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:51.959777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:52.006675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c43b45-1336-4732-98af-a105f08b7cf7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:21:52.083284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:21:52.083892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:52.084166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:52.084433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:52.086018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:52.193910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:21:52.197965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:21:52.200116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2870a3-af7a-41d1-8bea-10ad2ee8aa8c service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:21:52.218050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 684/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:21:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:21:57.121703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759315c-c704-4fd5-91fd-2dcd78e61eee None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:57.122145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759315c-c704-4fd5-91fd-2dcd78e61eee None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:57.122240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759315c-c704-4fd5-91fd-2dcd78e61eee None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:57.123861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 666/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:21:57.131446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991232-f1c8-44cc-9d35-572108550f33 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:21:57.131950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991232-f1c8-44cc-9d35-572108550f33 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:21:57.132037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991232-f1c8-44cc-9d35-572108550f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:21:57.266095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a9991232-f1c8-44cc-9d35-572108550f33 None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:21:57.267584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a9991232-f1c8-44cc-9d35-572108550f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:21:57.421513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 685/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:21:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:01.508991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723d02b-f577-4522-8178-eef34a997e65 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:01.509833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723d02b-f577-4522-8178-eef34a997e65 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:01.510114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723d02b-f577-4522-8178-eef34a997e65 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:01.511621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 667/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:01.518318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec99891-a8b0-4cff-a4a6-36e0f55a9d2b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:01.518827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec99891-a8b0-4cff-a4a6-36e0f55a9d2b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:01.519292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec99891-a8b0-4cff-a4a6-36e0f55a9d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:01.578469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:01.600841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-aec99891-a8b0-4cff-a4a6-36e0f55a9d2b None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:01.602129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aec99891-a8b0-4cff-a4a6-36e0f55a9d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:01.616892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=59 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:01.719155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 686/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:22:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:02.990632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2358fd-21ba-4bdd-a8ae-f0454a97e1e4 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:02.991206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2358fd-21ba-4bdd-a8ae-f0454a97e1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:02.991206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2358fd-21ba-4bdd-a8ae-f0454a97e1e4 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:02.992405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 668/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:02.997701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fd5de-db08-4a6b-b78e-02ea2bd0a2ac None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:02.998157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fd5de-db08-4a6b-b78e-02ea2bd0a2ac None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:02.998273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fd5de-db08-4a6b-b78e-02ea2bd0a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:03.074895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8f7fd5de-db08-4a6b-b78e-02ea2bd0a2ac None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:03.076099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8f7fd5de-db08-4a6b-b78e-02ea2bd0a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:03.119990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:03.187214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 687/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:22:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:06.597713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebc81-b78d-48ca-9d12-cc034639796c None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:06.598209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebc81-b78d-48ca-9d12-cc034639796c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:06.598309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ebc81-b78d-48ca-9d12-cc034639796c None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:06.599676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 669/1356] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:06.604889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a17233-ab9e-4b3e-91bd-1ac4191d3d4c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:06.605329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a17233-ab9e-4b3e-91bd-1ac4191d3d4c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:06.605437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a17233-ab9e-4b3e-91bd-1ac4191d3d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:06.691022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-26a17233-ab9e-4b3e-91bd-1ac4191d3d4c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:06.692227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-26a17233-ab9e-4b3e-91bd-1ac4191d3d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:06.819403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 688/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:07.122970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:07.123563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:07.129807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:07.129807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 670/1358] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 05:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:07.133280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718046e3-14e9-4168-83f6-0e639b1412e2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:07.133568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718046e3-14e9-4168-83f6-0e639b1412e2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:07.133841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718046e3-14e9-4168-83f6-0e639b1412e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:07.151537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:07.220096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-718046e3-14e9-4168-83f6-0e639b1412e2 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:07.221275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-718046e3-14e9-4168-83f6-0e639b1412e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:07.354181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 689/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 05:22:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:07.369608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:07.372318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:07.522961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a58ddb-70f5-454c-aac8-f2ebf0e39bbf None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:07.597918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:07.652424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:07.653022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:07.653486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:07.653761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:07.655388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:07.748240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:07.749842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad76a45-ae06-4e00-b194-1c3db781fc80 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:07.768078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 671/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:09.882556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:09.883032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:09.883110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:09.884619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 690/1361] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:09.890369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0387e8-c5e4-4ef0-9a7c-4eca89e537d3 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:09.890864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0387e8-c5e4-4ef0-9a7c-4eca89e537d3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:09.890996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0387e8-c5e4-4ef0-9a7c-4eca89e537d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:09.963211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cb0387e8-c5e4-4ef0-9a7c-4eca89e537d3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:09.965147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cb0387e8-c5e4-4ef0-9a7c-4eca89e537d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:10.006545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:10.069500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 672/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:10.357767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:10.359539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:10.426353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:10.511924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4403e58-426b-4c54-bbcb-4eca763ec937 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:10.631297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:10.631918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:10.632195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:10.632463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:10.634093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:10.659508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:10.731825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:10.734298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972133da-646a-4deb-943f-5ccdff96e65b service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:10.757538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 691/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:13.124115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfec5-21ce-409d-ba69-cd73d70ff024 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:13.124701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfec5-21ce-409d-ba69-cd73d70ff024 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:13.124701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cfec5-21ce-409d-ba69-cd73d70ff024 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:13.132867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 673/1364] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:13.133022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1527a8a-3605-4bd1-adfc-427d832db89a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:13.133022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1527a8a-3605-4bd1-adfc-427d832db89a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:13.133022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1527a8a-3605-4bd1-adfc-427d832db89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:13.206715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c1527a8a-3605-4bd1-adfc-427d832db89a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:13.207931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c1527a8a-3605-4bd1-adfc-427d832db89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:13.306927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:13.310583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 692/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:13.602025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:13.602550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:13.602950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:13.605144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 674/1366] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 05:22:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:13.610987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fdacd-a682-4e92-bf9a-c01681c57b6b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:13.616213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fdacd-a682-4e92-bf9a-c01681c57b6b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:13.616213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fdacd-a682-4e92-bf9a-c01681c57b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:13.697736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9a7fdacd-a682-4e92-bf9a-c01681c57b6b None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:13.698903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9a7fdacd-a682-4e92-bf9a-c01681c57b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:13.819186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 693/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 05:22:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:13.827293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:13.829178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:13.997660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab589437-3576-430d-8d80-a26c40566d3f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:14.020072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:14.120947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:14.121810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:14.122284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:14.122672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:14.124352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:14.227003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:14.228626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d65549-f3e2-48d4-bdb5-ae349807e793 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:14.248195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 675/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:14.779965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253301-45a5-48e0-9663-6faee867a59c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:14.780536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253301-45a5-48e0-9663-6faee867a59c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:14.780536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253301-45a5-48e0-9663-6faee867a59c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:14.782002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 694/1369] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:14.787010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99342-088d-4982-953e-f408ddd32796 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:14.787294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99342-088d-4982-953e-f408ddd32796 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:14.787566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99342-088d-4982-953e-f408ddd32796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:14.821206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:14.857766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-6de99342-088d-4982-953e-f408ddd32796 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:14.859272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6de99342-088d-4982-953e-f408ddd32796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:14.962914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 676/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:15.547841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:15.547841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:15.547841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:15.550332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 695/1371] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 1 05:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:15.554490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9205-b322-4d16-8290-f88eda38b5ba None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:15.554490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9205-b322-4d16-8290-f88eda38b5ba None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:15.554490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd9205-b322-4d16-8290-f88eda38b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:15.630732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f3cd9205-b322-4d16-8290-f88eda38b5ba None None] MFA Rules not processed for user `b6cb1f27ea5c49c9a805a9cdd678bef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:15.631964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f3cd9205-b322-4d16-8290-f88eda38b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:15.686178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:15.719655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 677/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 1 05:22:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:15.727334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:15.729493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:15.784667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:15.843123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61399f-f9c7-4b97-9a8f-241c0b4bad2d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:15.923731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:15.924353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:15.924632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:15.924907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:15.926587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:16.028817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:16.045952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:16.047757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b2b58-206e-418e-b19a-89901ac7b0ca service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:16.070298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 696/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:16.345578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:16.345899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:16.346159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:16.347725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 678/1374] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:16.353111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d20233-4827-41b0-901e-1e6378d4b4b5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:16.353709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d20233-4827-41b0-901e-1e6378d4b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:16.354198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d20233-4827-41b0-901e-1e6378d4b4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:16.423018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-60d20233-4827-41b0-901e-1e6378d4b4b5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:16.424328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60d20233-4827-41b0-901e-1e6378d4b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:16.519093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 697/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:16.805031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:16.806586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:16.956950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f3ba7-c6d4-40d8-8dc3-e3405207a02d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:17.007306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:17.068367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:17.068966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:17.069243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:17.069513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:17.071169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:17.169988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:17.171255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d061c-b636-4c8b-9d0c-acec4973c000 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:17.190073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 679/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:19.262512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:19.262999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:19.262999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:19.264288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 698/1377] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:19.269534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d397-e156-4840-971f-a5ce96113802 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:19.269951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d397-e156-4840-971f-a5ce96113802 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:19.270076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5d397-e156-4840-971f-a5ce96113802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:19.334123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-bca5d397-e156-4840-971f-a5ce96113802 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:19.335342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bca5d397-e156-4840-971f-a5ce96113802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:19.341903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:19.433737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 680/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:19.726663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:19.728163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:19.795075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:19.884943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f74055-cda2-4c7f-9897-3bb11667d6ca None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:19.999707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:20.000226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:20.000503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:20.000772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:20.002339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:20.020169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:20.094175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:20.095924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d58c22-da04-43ec-9f86-6065b03868ae service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:20.114594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 699/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:22.225912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08525d-f929-48d1-a358-11990c826354 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:22.226463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08525d-f929-48d1-a358-11990c826354 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:22.226463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08525d-f929-48d1-a358-11990c826354 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:22.227893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 681/1380] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:22.232952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0ec2-60d7-4a32-902a-e501a62f1948 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:22.233654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0ec2-60d7-4a32-902a-e501a62f1948 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:22.233739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b0ec2-60d7-4a32-902a-e501a62f1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:22.303143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-048b0ec2-60d7-4a32-902a-e501a62f1948 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:22.304468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-048b0ec2-60d7-4a32-902a-e501a62f1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:22.411316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:22.419221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 700/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:22.708884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08525d-f929-48d1-a358-11990c826354 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:22.710393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ee08525d-f929-48d1-a358-11990c826354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:22.851887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:22.863760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08525d-f929-48d1-a358-11990c826354 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:22.985395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:22.985930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:22.986207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:22.986522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:22.988208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:23.083887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:23.085896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245e0d-b2bb-4fd7-a274-4688d31cb838 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:23.097074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:23.106710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 682/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:25.557965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:25.558543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:25.558543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:25.559931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 701/1383] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:25.565468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc351e5-09fa-48fe-b5a8-5ec834e0a082 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:25.565989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc351e5-09fa-48fe-b5a8-5ec834e0a082 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:25.565989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc351e5-09fa-48fe-b5a8-5ec834e0a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:25.637382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ecc351e5-09fa-48fe-b5a8-5ec834e0a082 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:25.638640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ecc351e5-09fa-48fe-b5a8-5ec834e0a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:25.736315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 683/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:26.032885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:26.034893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:26.193724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffe854-c618-4e7b-930a-024d8533de7a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:26.272730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:26.322375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:26.323040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:26.323334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:26.323618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:26.325301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:26.424726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:26.426331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c2112-b8ce-45f5-bf4d-523db52b8ac4 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:26.444152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 702/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:28.431512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:28.432062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:28.432309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:28.434145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 684/1386] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:28.439308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949eeb1-dd54-4dda-9698-55f538d85fab None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:28.439707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949eeb1-dd54-4dda-9698-55f538d85fab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:28.439880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949eeb1-dd54-4dda-9698-55f538d85fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:28.510045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9949eeb1-dd54-4dda-9698-55f538d85fab None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:28.511359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9949eeb1-dd54-4dda-9698-55f538d85fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:28.546378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:28.611807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 703/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:28.916618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:28.918437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:29.001156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:29.072801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cc3a4-02eb-4ff2-bc87-87c954c6c53e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:29.190953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:29.191572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:29.191848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:29.192115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:29.193712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:29.233944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:29.291012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:29.292704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1c6a1-ef8c-4bf2-8a88-1922c66ec53d service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:29.311438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 685/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:31.318245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93ead-4f6f-45d8-99e4-59a87f600b70 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.318843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93ead-4f6f-45d8-99e4-59a87f600b70 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.318947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93ead-4f6f-45d8-99e4-59a87f600b70 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.320232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 704/1389] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:31.326251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3481d-13c2-41a7-96fe-503cc5a239e9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.326759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3481d-13c2-41a7-96fe-503cc5a239e9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.326996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3481d-13c2-41a7-96fe-503cc5a239e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.362099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93cf3-2bb6-493e-8cee-5cc2760fbc31 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.362384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93cf3-2bb6-493e-8cee-5cc2760fbc31 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.362706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb93cf3-2bb6-493e-8cee-5cc2760fbc31 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.364104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 705/1390] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:22:31.369185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ce4f8-b2cf-4618-bcc4-9735b8b7eab8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.369466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ce4f8-b2cf-4618-bcc4-9735b8b7eab8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.369741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ce4f8-b2cf-4618-bcc4-9735b8b7eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.395495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-37c3481d-13c2-41a7-96fe-503cc5a239e9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:31.396724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-37c3481d-13c2-41a7-96fe-503cc5a239e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:31.435752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-720ce4f8-b2cf-4618-bcc4-9735b8b7eab8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:31.436861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-720ce4f8-b2cf-4618-bcc4-9735b8b7eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:31.498572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 686/1391] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:31.539137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 706/1392] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:31.799965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.800347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.800686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.802144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 687/1393] 10.222.0.16 () {54 vars in 967 bytes} [Thu Sep 1 05:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:31.807953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:31.808304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:31.808646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:31.825179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:31.826361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:31.827960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:31.841897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:31.889471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] MFA Rules not processed for user `fc91470011c746488ea56ca5d9e7a8e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:31.890712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:31.967348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 707/1394] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Sep 1 05:22:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:31.973242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ad893-dda0-4c73-a474-2b80d5ca86b4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:31.974846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:31.982052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eddfe-d1c5-4746-b707-3c53f6f63132 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:32.054270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:32.054814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:32.055087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:32.055353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:32.057458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:32.060347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:32.086990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:32.087632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:32.087754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:32.087916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:32.088195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:32.089825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:32.170005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:32.171550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e36c99-53c8-469b-8c53-7519bd19cada service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:32.183600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:32.185195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d2ea6-f80d-4353-99a3-ec477a3148b9 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:32.190796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 708/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:22:32.204514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 688/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:33.098650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c600-a6c1-43ac-b5ca-f09776f3b8c6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.099111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c600-a6c1-43ac-b5ca-f09776f3b8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.099334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4c600-a6c1-43ac-b5ca-f09776f3b8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.166010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e3b4c600-a6c1-43ac-b5ca-f09776f3b8c6 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:33.167297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e3b4c600-a6c1-43ac-b5ca-f09776f3b8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:33.250740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:33.256782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 709/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:33.300807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.301292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.301492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.303168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 689/1398] 10.222.0.16 () {54 vars in 943 bytes} [Thu Sep 1 05:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:33.309657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401342-b1a6-41bb-8d88-c30a056442dd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.310041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401342-b1a6-41bb-8d88-c30a056442dd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.310439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95401342-b1a6-41bb-8d88-c30a056442dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.387330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-95401342-b1a6-41bb-8d88-c30a056442dd None None] MFA Rules not processed for user `01a4e7f4d1e54b17938c3514428a34b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:33.388539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-95401342-b1a6-41bb-8d88-c30a056442dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:33.496071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 710/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Thu Sep 1 05:22:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:33.505772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:33.507396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:33.612210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:33.656909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030e335-ecae-4a90-8533-08c99f3fc51c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:33.829203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:33.829203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:33.829203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:33.829203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:33.829885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:33.855881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:33.857859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2c5e5-9d03-4115-96ca-3e909a8298a2 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:33.861536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:33.866673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 690/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:22:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:22:34.263959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:34.264401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:34.264512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:34.265886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 711/1401] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:34.271540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f4bf-db36-4d06-9e59-fa52b2fe623f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:34.271762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f4bf-db36-4d06-9e59-fa52b2fe623f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:34.271963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6f4bf-db36-4d06-9e59-fa52b2fe623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:34.338031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-59d6f4bf-db36-4d06-9e59-fa52b2fe623f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:34.339315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-59d6f4bf-db36-4d06-9e59-fa52b2fe623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:34.439224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 691/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:34.728746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:34.730266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:34.795756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:34.886959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6a1ee-1206-4353-bad4-fce696e686da None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:35.011939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:35.012464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:35.012741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:35.013010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:35.014649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:35.032991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:35.116160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:35.117863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ae49b-6f41-4bfd-880e-cae1b8becde1 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:35.139233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 712/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:37.243929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:37.244476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:37.244476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:37.245983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 692/1404] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:37.251365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2b57-2acf-405e-80f8-435691cf44fd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:37.251880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2b57-2acf-405e-80f8-435691cf44fd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:37.251880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d2b57-2acf-405e-80f8-435691cf44fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:37.338693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a67d2b57-2acf-405e-80f8-435691cf44fd None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:37.339910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a67d2b57-2acf-405e-80f8-435691cf44fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:37.450183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:37.466558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 713/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:37.748595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:37.750212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:37.834995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:37.920086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13b29d-a903-44f5-ae5f-62dc55a222c2 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:38.057014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:38.057624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:38.057899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:38.058170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:38.070690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:38.105002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:38.165542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:38.167301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f4ec7-1480-4055-8984-30fad84ed990 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:38.191897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 693/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:41.086436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:41.087618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:41.087714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:41.089082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 714/1407] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:41.094396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211b8a-55d1-445e-a729-8415d1983c84 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:41.094841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211b8a-55d1-445e-a729-8415d1983c84 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:41.095041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00211b8a-55d1-445e-a729-8415d1983c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:41.166170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-00211b8a-55d1-445e-a729-8415d1983c84 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:41.167432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-00211b8a-55d1-445e-a729-8415d1983c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:41.288466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 694/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:22:41.573967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:22:41.575895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:41.759634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ade58-854b-4169-b9b5-bc87c0da80c3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:22:41.833929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:41.889915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:22:41.890574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:41.890858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:41.891143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:41.892753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:41.996719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:22:41.998378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e14ec-da71-4d9f-807d-eda084f8088a service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:22:42.018668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 715/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:22:51.280198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=93 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:52.015229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=65 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:57.378544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683eefb-7664-4662-b7a5-1d02e49599a3 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:57.379187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683eefb-7664-4662-b7a5-1d02e49599a3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:57.379187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683eefb-7664-4662-b7a5-1d02e49599a3 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:57.380458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 695/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 1 05:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:22:57.385651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332040f-ce76-4ff4-9158-d26fa76713ce None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:57.386122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332040f-ce76-4ff4-9158-d26fa76713ce None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:57.386222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332040f-ce76-4ff4-9158-d26fa76713ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:57.461334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a332040f-ce76-4ff4-9158-d26fa76713ce None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:57.462510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a332040f-ce76-4ff4-9158-d26fa76713ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:57.567479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 716/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 1 05:22:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:22:57.582500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4f9-4baa-4681-8863-d3ff2eabdf98 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:57.582785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4f9-4baa-4681-8863-d3ff2eabdf98 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:57.583054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2be4f9-4baa-4681-8863-d3ff2eabdf98 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:57.584390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 696/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 1 05:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:22:57.589578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ed89-b004-45e0-a1f9-8ea988bfffe9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:22:57.589860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ed89-b004-45e0-a1f9-8ea988bfffe9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:22:57.590126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0ed89-b004-45e0-a1f9-8ea988bfffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:22:57.661484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:22:57.664235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-54e0ed89-b004-45e0-a1f9-8ea988bfffe9 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:22:57.665429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-54e0ed89-b004-45e0-a1f9-8ea988bfffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:22:57.778528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 717/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 1 05:22:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:00.131141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:00.131676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:00.131676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:00.133513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:00.138730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e05d6-b366-426c-80a5-1b11a9d365a9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:00.139174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e05d6-b366-426c-80a5-1b11a9d365a9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:00.143954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e05d6-b366-426c-80a5-1b11a9d365a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:00.204476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-607e05d6-b366-426c-80a5-1b11a9d365a9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:00.205670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-607e05d6-b366-426c-80a5-1b11a9d365a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:00.259398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:00.299596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:00.585816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:00.587461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:00.732302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e066a-a119-46fd-9a9c-e6dd40c59940 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:00.802145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:00.842077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:00.842653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:00.842937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:00.843210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:00.844809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:00.932643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:00.934257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aece22c-e06e-4dc0-a09f-8eb0669deef8 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:00.951726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 698/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:01.838136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:01.838723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:01.838723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:01.840047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:01.844951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfae-8983-430a-a6d6-d96f3e111cee None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:01.845409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfae-8983-430a-a6d6-d96f3e111cee None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:01.845509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3acfae-8983-430a-a6d6-d96f3e111cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:01.921200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8f3acfae-8983-430a-a6d6-d96f3e111cee None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:01.922319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8f3acfae-8983-430a-a6d6-d96f3e111cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:01.948576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:02.018035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:02.169419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:02.170977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:02.432105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce26ac2-0851-4bd7-824d-34637ab9f876 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:02.471906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:02.544295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:02.544903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:02.545181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:02.545477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:02.547112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:02.633841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:02.635529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee80f2-851e-441a-b446-3dc94a1c1bde service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:02.653948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 720/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:03.522196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:03.522830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:03.522830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:03.524145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:03.528933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384a151-6f7d-45b9-9610-346387dadbd1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:03.529326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384a151-6f7d-45b9-9610-346387dadbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:03.529459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384a151-6f7d-45b9-9610-346387dadbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:03.582085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:03.603273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8384a151-6f7d-45b9-9610-346387dadbd1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:03.604466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8384a151-6f7d-45b9-9610-346387dadbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:03.698110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:03.846816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:03.848253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:03.991847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f1c79-63b6-4022-9e41-697495b25d28 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:04.003622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:04.102235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:04.102775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:04.103048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:04.103312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:04.104886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:04.190811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:04.192426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b087523-c52f-4648-ba9d-c9c9730dbb2c service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:04.209723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 701/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:05.058337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477bfa-c616-4340-b810-7b7732174026 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:05.058905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477bfa-c616-4340-b810-7b7732174026 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:05.058905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90477bfa-c616-4340-b810-7b7732174026 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:05.060241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:05.065202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee49c-9425-48ea-8da0-3b1eacbd2e57 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:05.065670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee49c-9425-48ea-8da0-3b1eacbd2e57 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:05.065779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee49c-9425-48ea-8da0-3b1eacbd2e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:05.086139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:05.140982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8a5ee49c-9425-48ea-8da0-3b1eacbd2e57 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:05.142220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8a5ee49c-9425-48ea-8da0-3b1eacbd2e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:05.236671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:05.388971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90477bfa-c616-4340-b810-7b7732174026 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:05.390520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-90477bfa-c616-4340-b810-7b7732174026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:05.498944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:05.533911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90477bfa-c616-4340-b810-7b7732174026 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:05.644300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:05.644904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:05.645176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:05.645436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:05.647030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:05.721068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:05.735270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:05.736858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e882ad8-ca05-4c62-a1fe-c747ced6a625 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:05.754134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 723/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:06.602353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:06.602878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:06.602973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:06.604277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:06.609858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e52d0-0f07-453d-8b41-eb221ec9e373 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:06.610299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e52d0-0f07-453d-8b41-eb221ec9e373 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:06.610393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e52d0-0f07-453d-8b41-eb221ec9e373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:06.686394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-533e52d0-0f07-453d-8b41-eb221ec9e373 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:06.687649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-533e52d0-0f07-453d-8b41-eb221ec9e373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:06.781350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:06.931491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:06.933014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:07.015739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:07.078883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc036bf-90e3-42ca-9228-b950eb9f787e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:07.189770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:07.190397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:07.190707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:07.190976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:07.192563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:07.242259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:07.282951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:07.284581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e423fda-7f33-4533-9649-94ba5793ebfe service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:07.303204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 704/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:08.138932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:08.139511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:08.139511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:08.140810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:08.145692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1633-f3d6-4989-bb2d-3b0e3625a05f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:08.146119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1633-f3d6-4989-bb2d-3b0e3625a05f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:08.146337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a1633-f3d6-4989-bb2d-3b0e3625a05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:08.220879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cc3a1633-f3d6-4989-bb2d-3b0e3625a05f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:08.222083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cc3a1633-f3d6-4989-bb2d-3b0e3625a05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:08.315821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:08.463105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:08.464552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:08.513384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:08.608304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ae99-f9a0-4f8e-9e6e-dc842d689f60 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:08.719624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:08.720162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:08.720481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:08.720752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:08.722328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:08.729766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:08.809704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:08.811329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c749413-b519-4943-a9c8-9058fe400417 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:08.829617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 726/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:09.712217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:09.712882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:09.712882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:09.714146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:09.719165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f56c-3e36-4ca3-b32a-4557a6a235c9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:09.719628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f56c-3e36-4ca3-b32a-4557a6a235c9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:09.719744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f56c-3e36-4ca3-b32a-4557a6a235c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:09.797602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c365f56c-3e36-4ca3-b32a-4557a6a235c9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:09.798930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c365f56c-3e36-4ca3-b32a-4557a6a235c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:09.864402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:09.893750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:10.044821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:10.046459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:10.063748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:10.192241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeb919-fd86-4233-9f87-16c70f155ec8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:10.276817 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:10.301825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:10.302442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:10.302719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:10.302986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:10.304606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:10.391006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:10.392623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b1b55-2212-48bf-a51f-f52d25b05a1f service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:10.410374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 707/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:11.277046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:11.277608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:11.277608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:11.278952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:11.283879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3b550-173c-448b-8bc3-9148c2714bad None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:11.284282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3b550-173c-448b-8bc3-9148c2714bad None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:11.284422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3b550-173c-448b-8bc3-9148c2714bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:11.357986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-14d3b550-173c-448b-8bc3-9148c2714bad None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:11.359200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-14d3b550-173c-448b-8bc3-9148c2714bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:11.400552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:11.455461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:11.607348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:11.608975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:11.766556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e3683-0a19-4e9a-90b4-f2dec96e27d6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:11.819874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:11.878498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:11.879131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:11.879402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:11.879671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:11.881247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:11.967170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:11.968781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af30559-e022-408b-b85e-36f14d1a605a service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:11.987003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 729/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:12.838770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:12.839351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:12.839351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:12.840653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:12.845507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defdadc-eb22-4d11-8d16-3b98872086dd None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:12.845943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defdadc-eb22-4d11-8d16-3b98872086dd None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:12.846054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defdadc-eb22-4d11-8d16-3b98872086dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:12.919733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3defdadc-eb22-4d11-8d16-3b98872086dd None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:12.920846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3defdadc-eb22-4d11-8d16-3b98872086dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:12.922860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:13.018640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:13.166466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:13.167998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:13.314015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bc482-7e91-4d7f-9ae2-bafb0566977c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:13.341202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:13.426111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:13.426854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:13.427136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:13.427409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:13.429632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:13.516023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:13.517550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473a93d-80f4-4f0b-939e-8727300b3a91 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:13.535080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 710/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:15.341848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:15.342467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:15.342467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:15.343757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:15.348859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65676-2053-49ea-9020-cabf81f2b52c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:15.349275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65676-2053-49ea-9020-cabf81f2b52c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:15.349410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65676-2053-49ea-9020-cabf81f2b52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:15.391372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:15.416339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c2b65676-2053-49ea-9020-cabf81f2b52c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:15.417558 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c2b65676-2053-49ea-9020-cabf81f2b52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:15.514778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:15.797047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:15.798615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:15.900175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071c79e-56bb-4352-8f9d-6a4680531fcc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:15.932087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:15.972040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:15.972817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:15.973268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:15.973584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:15.976207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:16.080814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:16.082199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40209a-d8c9-4365-8119-768f36822ca0 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:16.101231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 732/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:18.140106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:18.140658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:18.140658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:18.142046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:18.148134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae1952-e169-43b4-8d22-2238bf962ae8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:18.148598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae1952-e169-43b4-8d22-2238bf962ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:18.148698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ae1952-e169-43b4-8d22-2238bf962ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:18.213153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-05ae1952-e169-43b4-8d22-2238bf962ae8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:18.214354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05ae1952-e169-43b4-8d22-2238bf962ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:18.226775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:18.309211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:18.589017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:18.590589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:18.695636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c96ac-d2e1-4440-8107-1ddbf717ee20 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:18.707910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:18.771088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:18.771698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:18.772036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:18.772303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:18.773899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:18.880331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:18.881736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5747e-71cc-4553-bcbe-1dbc26fa6f9d service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:18.899112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 713/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:21.148414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:21.148991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:21.148991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:21.150319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:21.155570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc898a8-d981-4398-98b5-e8db2cc9c913 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:21.156019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc898a8-d981-4398-98b5-e8db2cc9c913 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:21.156136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc898a8-d981-4398-98b5-e8db2cc9c913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:21.203541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:21.221356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-bbc898a8-d981-4398-98b5-e8db2cc9c913 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:21.222619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bbc898a8-d981-4398-98b5-e8db2cc9c913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:21.316925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:21.601137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:21.602710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:21.707733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f6da9-15d3-42e8-a3c9-db1630a53291 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:21.748040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:21.781774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:21.782373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:21.782672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:21.782935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:21.784510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:21.890884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:21.892239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde89947-d6fc-4b16-be62-c498044cd73c service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:21.910318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 735/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:23.906043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:23.906605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:23.906605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:23.907950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:23.913063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a4118-4d0d-4cfe-a128-99f1122f59b8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:23.913489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a4118-4d0d-4cfe-a128-99f1122f59b8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:23.913611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a4118-4d0d-4cfe-a128-99f1122f59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:23.977237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-340a4118-4d0d-4cfe-a128-99f1122f59b8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:23.978456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-340a4118-4d0d-4cfe-a128-99f1122f59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:23.998370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:24.071921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:24.358363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:24.359977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:24.462091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d33a-0370-402e-829d-fa2809d8cd99 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:24.481374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:24.533379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:24.533969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:24.534241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:24.534532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:24.536114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:24.644385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:24.645872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f5988-fc90-46ed-ba4b-e3fdbfe6c344 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:24.664160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 716/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:26.701541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:26.702098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:26.702236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:26.703564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:26.709085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9816-91b1-47a2-b24f-487d3009626c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:26.709515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9816-91b1-47a2-b24f-487d3009626c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:26.709620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df9816-91b1-47a2-b24f-487d3009626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:26.772775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:26.775422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-18df9816-91b1-47a2-b24f-487d3009626c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:26.776623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-18df9816-91b1-47a2-b24f-487d3009626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:26.881651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:23:27.164906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:27.166466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:27.272504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2692b6-3b60-4e6a-ad17-0c844e02cce0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:27.323286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:27.365061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:27.365870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:27.366146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:27.366432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:27.368048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:27.490033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:27.491591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de63ac4-e36f-42d0-a592-b789b7b3ec79 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:27.518943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 738/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:23:29.532165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:29.532683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:29.532683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:29.534043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:29.539811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682b0e2-2292-4196-a5f4-c9c39f2cdbb0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:29.540204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682b0e2-2292-4196-a5f4-c9c39f2cdbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:29.540381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682b0e2-2292-4196-a5f4-c9c39f2cdbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:29.605694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a682b0e2-2292-4196-a5f4-c9c39f2cdbb0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:29.606941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a682b0e2-2292-4196-a5f4-c9c39f2cdbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:29.635759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:29.702823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:29.996237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:29.997812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:30.100282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba1bc1-0284-439d-9b75-f3c2cdf30252 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:30.127753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:30.172358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:30.172955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:30.173235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:30.173510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:30.175158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:30.281177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:30.282641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d08b0-db88-4bf9-931f-3cf160a54658 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:30.301310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 719/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:32.245874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.246477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.246477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.247828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:32.253049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff770-e866-4bce-a069-b128ce31e354 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.253480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff770-e866-4bce-a069-b128ce31e354 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.253577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ff770-e866-4bce-a069-b128ce31e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.318666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c90ff770-e866-4bce-a069-b128ce31e354 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:32.319862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c90ff770-e866-4bce-a069-b128ce31e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.327949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:32.415602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:32.697165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:32.698746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.799954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c025e56-77d4-42f9-9d77-b19ea9396e60 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:32.854924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:32.873006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:32.873652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:32.873944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:32.874507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:32.876531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:32.982645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:32.984001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cb850-76e3-4e53-bee7-5fad76894ee7 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:33.002366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 741/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:34.937711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:34.938249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:34.938249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:34.939568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:34.944782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4deff8-5b12-4591-9dd6-5fa0bea2961c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:34.945212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4deff8-5b12-4591-9dd6-5fa0bea2961c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:34.945431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4deff8-5b12-4591-9dd6-5fa0bea2961c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:35.010585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8c4deff8-5b12-4591-9dd6-5fa0bea2961c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:35.011817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8c4deff8-5b12-4591-9dd6-5fa0bea2961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:35.049932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:35.106896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:35.391051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:35.392602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:35.496577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a53b69f-60ec-4fd9-b31d-bbf08587caa7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:35.533306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:35.570369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:35.570996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:35.571269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:35.571536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:35.573118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:35.678953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:35.680395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178989-141f-44cf-8d98-36a119f5619b service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:35.698806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 722/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:37.601895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:37.602517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:37.602517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:37.603854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:37.609201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f94dc-8148-4e15-9d5c-0483c351893a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:37.609617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f94dc-8148-4e15-9d5c-0483c351893a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:37.609764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f94dc-8148-4e15-9d5c-0483c351893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:37.676828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-459f94dc-8148-4e15-9d5c-0483c351893a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:37.678039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-459f94dc-8148-4e15-9d5c-0483c351893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:37.695826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:37.773484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:38.054554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:38.056123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:38.157808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cf662-fd8e-4d56-be57-be873ed41893 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:38.223364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:38.230537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:38.231145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:38.231417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:38.231681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:38.233257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:38.340477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:38.341830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b163c0-c862-4a59-b2d3-c6c9d59f0669 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:38.360469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 744/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:40.255491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:40.256048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:40.256048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:40.257339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:23:40.262493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467a295-e44a-49c4-b390-8d2ef75875b4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:40.262904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467a295-e44a-49c4-b390-8d2ef75875b4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:40.263053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467a295-e44a-49c4-b390-8d2ef75875b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:40.328700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d467a295-e44a-49c4-b390-8d2ef75875b4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:23:40.329844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d467a295-e44a-49c4-b390-8d2ef75875b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:40.376467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:40.426147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:23:40.711582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:23:40.713126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:40.813140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcb33c-0a63-4caf-a357-4301e76f4e86 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:23:40.859321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:40.885596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:23:40.886129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:23:40.886428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:23:40.886699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:23:40.888279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:23:40.994494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:23:40.995941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c6de5-7c58-4c78-ac48-dff794fb3bb5 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:23:41.014366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 725/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:23:50.458157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=22 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:23:51.012130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=66 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:00.535797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9c89-6e92-44dd-81dc-75e0c45de82e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:00.536389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9c89-6e92-44dd-81dc-75e0c45de82e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:00.536389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9c89-6e92-44dd-81dc-75e0c45de82e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:00.537743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 746/1471] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:00.542835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a80c-2d4b-42bc-931d-f0463a03bda1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:00.543290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a80c-2d4b-42bc-931d-f0463a03bda1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:00.543404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09a80c-2d4b-42bc-931d-f0463a03bda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:00.607806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9e09a80c-2d4b-42bc-931d-f0463a03bda1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:00.609011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9e09a80c-2d4b-42bc-931d-f0463a03bda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:00.708081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 726/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:01.004944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:01.005223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:01.005497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:01.006873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 747/1473] 10.222.0.16 () {54 vars in 967 bytes} [Thu Sep 1 05:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:01.011661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cfc1-b9aa-4f54-ab6c-78c71f85e216 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:01.011950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cfc1-b9aa-4f54-ab6c-78c71f85e216 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:01.012221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180cfc1-b9aa-4f54-ab6c-78c71f85e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:01.086290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4180cfc1-b9aa-4f54-ab6c-78c71f85e216 None None] MFA Rules not processed for user `fc91470011c746488ea56ca5d9e7a8e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:01.087588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4180cfc1-b9aa-4f54-ab6c-78c71f85e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:01.107879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:01.163179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 727/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Sep 1 05:24:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:01.170603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:01.172132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:01.274235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1919-b1e5-40b7-941c-f029560d535c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:01.345836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:01.346457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:01.346740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:01.347012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:01.349123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:01.401372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:01.455322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:01.456783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83b29d2-52b9-458e-aff4-d9df0c3c255f service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:01.476269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 748/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:24:03.464225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:03.464795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:03.464795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:03.466099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 728/1476] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:03.471278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fc42-2679-47d7-a987-bc1f729ca772 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:03.471734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fc42-2679-47d7-a987-bc1f729ca772 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:03.471840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100fc42-2679-47d7-a987-bc1f729ca772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:03.535660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7100fc42-2679-47d7-a987-bc1f729ca772 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:03.536857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7100fc42-2679-47d7-a987-bc1f729ca772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:03.630202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 749/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:03.912893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:03.914379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:04.037919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b3d7-d5cb-4256-a803-9f3bb71da553 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:04.085010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:04.128258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:04.128860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:04.129139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:04.129403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:04.130031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:04.131841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:24:04.131841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:24:04.131841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:24:04.138032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d9e28-1695-4434-bb73-87ef7d55c54e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:04.151622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 729/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 05:24:03 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:24:06.046904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:06.047479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:06.047479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:06.048820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 750/1479] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:06.054667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b42-096e-42fb-a308-fcddd77b24a5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:06.055117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b42-096e-42fb-a308-fcddd77b24a5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:06.055248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5b42-096e-42fb-a308-fcddd77b24a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:06.121234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d0db5b42-096e-42fb-a308-fcddd77b24a5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:06.122466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d0db5b42-096e-42fb-a308-fcddd77b24a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:06.216162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 730/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:06.512040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:06.513604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:06.523930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:06.657860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85542682-9b2e-4366-8fcf-2cb2908f963c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:06.736726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:06.768610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:06.769794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:06.770070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:06.770339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:06.771958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:06.857906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:06.859536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c3ebc-94d7-4f5d-a5b0-ae91f61c9129 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:06.876880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 751/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:08.958128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:08.958621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:08.958758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:08.960034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 731/1482] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:08.965255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dc52f-ac02-4809-9e3d-cd64163d7d23 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:08.968389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dc52f-ac02-4809-9e3d-cd64163d7d23 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:08.968389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dc52f-ac02-4809-9e3d-cd64163d7d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:09.033204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-461dc52f-ac02-4809-9e3d-cd64163d7d23 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:09.034956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-461dc52f-ac02-4809-9e3d-cd64163d7d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:09.081490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:09.131292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 752/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:09.415564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:09.417166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:09.440025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:09.520998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda5e0-79f6-4789-b532-95b644a73c96 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:09.593100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:09.593686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:09.593966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:09.594233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:09.595878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:09.668867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:09.703535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:09.704983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5601357-852e-4aef-80d8-c7fd2d794a56 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:09.723689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 732/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:11.649493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:11.649989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:11.650088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:11.651615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 753/1485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:11.656700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a766a-6dbb-493b-be43-6d4a1e37ad6f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:11.657161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a766a-6dbb-493b-be43-6d4a1e37ad6f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:11.657270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a766a-6dbb-493b-be43-6d4a1e37ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:11.723389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-423a766a-6dbb-493b-be43-6d4a1e37ad6f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:11.724608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-423a766a-6dbb-493b-be43-6d4a1e37ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:11.820196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 733/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:12.107259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:12.108793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:12.212611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4d6ae-ee35-4fde-9b11-08b27b5341ee None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:12.283649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:12.285513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:12.286303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:12.286486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:12.286763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:12.288378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:12.396550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:12.397915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3686b4a-d762-4de0-a301-a1786df03369 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:12.416270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 754/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:14.393644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:14.394217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:14.394217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:14.395674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 734/1488] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:14.400721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883b49-69f2-442f-b902-f9250ae5bfa4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:14.401135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883b49-69f2-442f-b902-f9250ae5bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:14.401245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883b49-69f2-442f-b902-f9250ae5bfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:14.466802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-97883b49-69f2-442f-b902-f9250ae5bfa4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:14.467994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97883b49-69f2-442f-b902-f9250ae5bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:14.520940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:14.563833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 755/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:14.852585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:14.854160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:14.889054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:14.955151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e876-e6aa-45e0-9ee2-28da4bf8f2a8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:15.026872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:15.027466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:15.027741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:15.028006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:15.029579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:15.107452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:15.136119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:15.137872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257125b1-ac67-4d3c-8beb-f09fe885552f service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:15.156114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 735/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:17.619493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:17.620094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:17.620094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:17.621393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 756/1491] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 05:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:24:17.626454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8ba09-eb30-454c-bf26-02ddfb9768ae None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:17.626909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8ba09-eb30-454c-bf26-02ddfb9768ae None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:17.627009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8ba09-eb30-454c-bf26-02ddfb9768ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:17.692764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-94b8ba09-eb30-454c-bf26-02ddfb9768ae None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:24:17.693975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-94b8ba09-eb30-454c-bf26-02ddfb9768ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:17.789939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 736/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 05:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:24:18.077471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:24:18.079134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:18.181759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92fcdd-79d1-484a-91c8-194dd7193330 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:24:18.254271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:24:18.254901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:24:18.255179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:24:18.255448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:24:18.257035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:24:18.265869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:18.364074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:24:18.365435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59a20e-89fc-43e0-a37a-29fc31af02e1 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:24:18.383893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 757/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:24:27.786985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=82 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:24:28.381573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=50 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:44.226818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.227409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.227409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:44.301104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:44.302321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:44.396616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 737/1494] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:44.404571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.405069 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.405180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:44.480411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:44.481538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:44.577403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 758/1495] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:44.582009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:44.583258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:44.594428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc92e1d-01ef-4ec5-badc-6e821c392ce4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:44.654369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:44.687098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:44.687704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.687981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.688250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:44.688893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:25:44.689194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:44.690877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:44.690877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:25:44.690877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:25:44.691757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:44.702879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 738/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:25:44.711279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:44.720290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f8883-5ae0-41b0-85ab-f6ade39d71a5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:44.782835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:44.816474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:44.817076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:44.817353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:44.817630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:44.818488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:44.820830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:44.820830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:25:44.820830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:25:44.822018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:44.968324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 759/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:25:44 2022] POST /identity/v3/projects => generated 377 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:25:44.973125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:44.974674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.011818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c374d-44d0-4d74-8e66-ca898339ce9d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:45.103677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:45.104208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.104485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.104754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.105384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:45.106398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:45.106398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:25:45.106398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:25:45.107630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:45.201464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 739/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:25:44 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:25:45.205853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:45.207503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.268316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ace709-575e-4752-bbb5-7ce6aecee209 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:45.349856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:45.359952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:45.360501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.360781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.361050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.361712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:45.362754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:45.362754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:25:45.362754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:25:45.363639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:45.375818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 760/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:25:45.379900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:45.381454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.393422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e8052e-e827-4578-bfe2-857d78fa2b63 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:45.419107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:45.489533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:45.490138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.490443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.490717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.491363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:45.492385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:45.492385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:25:45.492385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:25:45.493274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec34e05-1c5d-4643-b1a1-2fe4ca2dd4b3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:45.505849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 740/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:25:45.510240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:45.511845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.544817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466e0de-fa97-4d64-b95c-6151d42134c7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:45.638256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:45.639018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.639343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.639606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] PATH_INFO: `/v3/projects/c2688a094b02445696f42afb9a601789/users/75955208d72342c9b03b57da570723dc/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.661961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2688a094b02445696f42afb9a601789, user_id=75955208d72342c9b03b57da570723dc, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:45.663160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:45.663160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:25:45.663160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:25:45.665105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b99052-e9c4-4de9-a03d-3090cd185ccd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:45.773392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 761/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:25:45 2022] PUT /identity/v3/projects/c2688a094b02445696f42afb9a601789/users/75955208d72342c9b03b57da570723dc/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:25:45.785640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0cd6-b179-421e-b892-528a95a2771f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.785924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0cd6-b179-421e-b892-528a95a2771f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.786196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0cd6-b179-421e-b892-528a95a2771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.841509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-907d0cd6-b179-421e-b892-528a95a2771f None None] MFA Rules not processed for user `75955208d72342c9b03b57da570723dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:45.842731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-907d0cd6-b179-421e-b892-528a95a2771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:45.917904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 741/1502] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:25:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:45.931238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.931524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.931794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.933173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 762/1503] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 1 05:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:45.937924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e9292-663e-483e-bec6-f8237e08f372 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:45.938211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e9292-663e-483e-bec6-f8237e08f372 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:45.938526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e9292-663e-483e-bec6-f8237e08f372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:45.983022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:46.029746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-727e9292-663e-483e-bec6-f8237e08f372 None None] MFA Rules not processed for user `b6cb1f27ea5c49c9a805a9cdd678bef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:46.030993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-727e9292-663e-483e-bec6-f8237e08f372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:46.107104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 742/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 1 05:25:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:46.118315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:46.120585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:46.173082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:46.228349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916371a-f64b-454d-974d-306c894d89b3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:46.307669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:46.308278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:46.308553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:46.308819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:46.310434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:46.417140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:46.418616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:46.422088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:46.436873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 763/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:46.481730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9faf6f-4a80-4a2c-af29-3dda2c60640d None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:46.482156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9faf6f-4a80-4a2c-af29-3dda2c60640d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:46.482270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9faf6f-4a80-4a2c-af29-3dda2c60640d None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:46.483621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 743/1506] 10.222.0.16 () {52 vars in 830 bytes} [Thu Sep 1 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:25:46.487819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:46.489098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:46.587193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dab3fc-a451-4e7e-9178-978f92c25000 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:46.682004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:46.682657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:46.682931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:46.683199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] PATH_INFO: `/v3/users/75955208d72342c9b03b57da570723dc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:46.695492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75955208d72342c9b03b57da570723dc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:46.696479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:46.696479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:25:46.696479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:25:46.697712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:46.707786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:46.895308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75955208d72342c9b03b57da570723dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:46.919586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75955208d72342c9b03b57da570723dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:46.933160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75955208d72342c9b03b57da570723dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:46.933717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75955208d72342c9b03b57da570723dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:46.934139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75955208d72342c9b03b57da570723dc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:47.034964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 764/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:25:46 2022] DELETE /identity/v3/users/75955208d72342c9b03b57da570723dc => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:25:47.039313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:47.039597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:47.039867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:47.095397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:25:47.096602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:47.170278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:47.182497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 744/1508] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:25:47.192740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:47.193928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:47.298345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54966-b951-4559-97c5-0dfa911913c0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:47.349420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:47.368350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:47.368966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:47.369246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:47.369526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:47.371337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:47.483422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:47.484911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f7e8-2c8e-4abe-b442-f7ff5f2d3665 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:47.503991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 765/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:25:47.969339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:25:47.970802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:25:47.985809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c318cc-face-4635-a3a3-dd264bba6cb0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:25:48.082164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:25:48.082826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:25:48.083103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:25:48.083374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] PATH_INFO: `/v3/projects/c2688a094b02445696f42afb9a601789` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:25:48.092939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2688a094b02445696f42afb9a601789)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:25:48.093976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:25:48.093976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:25:48.093976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:25:48.095212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:25:48.222485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2688a094b02445696f42afb9a601789', 'request_id': 'req-e6d1a84e-040f-43df-ae45-e598d6928506'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:48.228578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2688a094b02445696f42afb9a601789', 'request_id': 'req-e6d1a84e-040f-43df-ae45-e598d6928506'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:48.286841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2688a094b02445696f42afb9a601789', 'request_id': 'req-e6d1a84e-040f-43df-ae45-e598d6928506'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:48.316856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] The token cache is being invalidate because project c2688a094b02445696f42afb9a601789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:25:48.317145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:48.317576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e6d1a84e-040f-43df-ae45-e598d6928506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:25:48.319014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 745/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:25:47 2022] DELETE /identity/v3/projects/c2688a094b02445696f42afb9a601789 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:25:57.501082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=62 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:25:58.314867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:17.838705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:17.839285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:17.839285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:17.840598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:17.845918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bf8b-dfce-4937-b0e2-c2235d88d325 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:17.846451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bf8b-dfce-4937-b0e2-c2235d88d325 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:17.846565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bf8b-dfce-4937-b0e2-c2235d88d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:17.923844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b3c7bf8b-dfce-4937-b0e2-c2235d88d325 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:17.924999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b3c7bf8b-dfce-4937-b0e2-c2235d88d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:18.031007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:18.184253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:18.186331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:18.306093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cd450d-56fb-431c-a498-f5d7112e2ca8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:18.311626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:18.312234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:18.312514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:18.312791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:18.321373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:18.322992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd265638-457a-4484-87d5-b6276f0c2282 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:18.332230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 767/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:27:19.171207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154518-40c1-42c5-9e60-a9f8af1f80c2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:19.171775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154518-40c1-42c5-9e60-a9f8af1f80c2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:19.171775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e154518-40c1-42c5-9e60-a9f8af1f80c2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:19.173140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:19.178576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c069-7544-4a7c-987e-eb8cd6e21958 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:19.179128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c069-7544-4a7c-987e-eb8cd6e21958 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:19.179128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c069-7544-4a7c-987e-eb8cd6e21958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:19.199534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f6d1c069-7544-4a7c-987e-eb8cd6e21958 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:19.200718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f6d1c069-7544-4a7c-987e-eb8cd6e21958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:19.208672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:19.359450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e154518-40c1-42c5-9e60-a9f8af1f80c2 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:19.374203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e154518-40c1-42c5-9e60-a9f8af1f80c2 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:19.380001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:19.380591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:19.380893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:19.381164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:19.389425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:19.391069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33545c-a237-4d33-bc41-03815d77eae3 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:19.400285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 748/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:20.686304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763288-02af-482c-95af-31ca80a6b5ef None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:20.686858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763288-02af-482c-95af-31ca80a6b5ef None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:20.686966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763288-02af-482c-95af-31ca80a6b5ef None None] PATH_INFO: `/v3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:20.688158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 769/1517] 10.222.0.60 () {54 vars in 925 bytes} [Thu Sep 1 05:27:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:27:20.694728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf33a-c984-4bab-a35a-6f5a0309e2fa None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:20.695149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf33a-c984-4bab-a35a-6f5a0309e2fa None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:20.695270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77cf33a-c984-4bab-a35a-6f5a0309e2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:20.871933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b77cf33a-c984-4bab-a35a-6f5a0309e2fa None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:20.873146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b77cf33a-c984-4bab-a35a-6f5a0309e2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:20.889226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:20.942173 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 749/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Sep 1 05:27:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:25.007535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7b-eeae-488d-b007-3762af4dbf21 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:25.008091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7b-eeae-488d-b007-3762af4dbf21 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:25.008091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7b-eeae-488d-b007-3762af4dbf21 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:25.009348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 770/1519] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 1 05:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:25.014306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e056c8-9573-40ce-9bfc-0f6bc8eb5bc0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:25.014756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e056c8-9573-40ce-9bfc-0f6bc8eb5bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:25.014879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e056c8-9573-40ce-9bfc-0f6bc8eb5bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:25.033761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a8e056c8-9573-40ce-9bfc-0f6bc8eb5bc0 None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:25.034965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a8e056c8-9573-40ce-9bfc-0f6bc8eb5bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:25.043533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 750/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 1 05:27:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:25.051381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870b7b-eeae-488d-b007-3762af4dbf21 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:25.065194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870b7b-eeae-488d-b007-3762af4dbf21 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:25.070254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:25.070875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:25.071156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:25.071431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:25.079746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:25.081327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160cde6-381c-4a24-9d46-c92d5d09435a service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:25.090788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 771/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:27:30.614544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527ecff-59ea-4387-a19c-ed516f7b4800 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:30.615099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527ecff-59ea-4387-a19c-ed516f7b4800 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:30.615218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527ecff-59ea-4387-a19c-ed516f7b4800 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:30.616559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:30.621401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a75cb-7c39-4988-9d6e-6c10c20418d7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:30.621889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a75cb-7c39-4988-9d6e-6c10c20418d7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:30.622027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a75cb-7c39-4988-9d6e-6c10c20418d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:30.641772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-757a75cb-7c39-4988-9d6e-6c10c20418d7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:30.643022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-757a75cb-7c39-4988-9d6e-6c10c20418d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:30.651573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:30.805084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527ecff-59ea-4387-a19c-ed516f7b4800 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:30.819943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527ecff-59ea-4387-a19c-ed516f7b4800 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:30.825369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:30.825960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:30.826230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:30.826528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:30.834761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:30.836345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b411b7-6c72-450c-8017-efbbb2c93c8e service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:30.845823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 752/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:31.666327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd001-97c1-49d1-9b9e-f17eb19eed8e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:31.666826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd001-97c1-49d1-9b9e-f17eb19eed8e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:31.666931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd001-97c1-49d1-9b9e-f17eb19eed8e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:31.668553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 773/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 05:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:31.673349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef795-a461-4548-b8d3-5400eb4836ad None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:31.673802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef795-a461-4548-b8d3-5400eb4836ad None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:31.673901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef795-a461-4548-b8d3-5400eb4836ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:31.693009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-849ef795-a461-4548-b8d3-5400eb4836ad None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:31.694201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-849ef795-a461-4548-b8d3-5400eb4836ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:31.702058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 753/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 05:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:31.853463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cd001-97c1-49d1-9b9e-f17eb19eed8e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:31.867454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cd001-97c1-49d1-9b9e-f17eb19eed8e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:31.872772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:31.873353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:31.873625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:31.873893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:31.882163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:31.883759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a481b7-b84e-4fb7-b0bc-657926779332 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:31.892912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 774/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:35.818639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:35.821451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:35.821451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:35.821451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:35.825720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d26fcd-9de4-4343-9348-c952d87d7470 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:35.826129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d26fcd-9de4-4343-9348-c952d87d7470 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:35.826263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d26fcd-9de4-4343-9348-c952d87d7470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:35.845773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-18d26fcd-9de4-4343-9348-c952d87d7470 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:35.847596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18d26fcd-9de4-4343-9348-c952d87d7470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:35.855402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:36.152503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:36.154168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:36.233497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bce9e8-0849-41ff-8a09-06d947fd5a8e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:36.237931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:36.238591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:36.238874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:36.239159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:36.248837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:36.250457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d32af-143f-414e-99b3-d680d6a6b5ad service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:36.261154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 755/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:38.144284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64deeb-4b68-44b0-b690-01454e488474 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:38.144866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64deeb-4b68-44b0-b690-01454e488474 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:38.144866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64deeb-4b68-44b0-b690-01454e488474 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:38.146178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:38.151389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a630-408c-4a02-b8a0-9aa6fc4951e0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:38.151798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a630-408c-4a02-b8a0-9aa6fc4951e0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:38.151919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559a630-408c-4a02-b8a0-9aa6fc4951e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:38.171439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-6559a630-408c-4a02-b8a0-9aa6fc4951e0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:38.172621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6559a630-408c-4a02-b8a0-9aa6fc4951e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:38.180816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:38.464357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64deeb-4b68-44b0-b690-01454e488474 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:38.475865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64deeb-4b68-44b0-b690-01454e488474 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:38.479358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:38.479979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:38.480255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:38.480522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:38.489220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:38.490663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6188e8-8971-421b-91c7-79dd851c6628 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:38.499882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 777/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:41.809712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:41.810284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:41.810284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:41.811639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:41.816675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ef13-0164-4202-bbf6-34666e6d8ee1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:41.817106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ef13-0164-4202-bbf6-34666e6d8ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:41.817201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8ef13-0164-4202-bbf6-34666e6d8ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:41.836100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0ed8ef13-0164-4202-bbf6-34666e6d8ee1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:41.837208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0ed8ef13-0164-4202-bbf6-34666e6d8ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:41.845048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:42.394607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:42.396405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:42.460165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:42.473131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af495b6-53d9-4067-9668-b47573c6db83 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:42.481967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:42.481967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:42.481967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:42.481967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:42.486498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:42.487942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f538137-a725-47bc-86f3-fd24f0ecf6a0 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:42.497828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 758/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:44.400753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:44.401331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:44.401331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:44.402673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 05:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:27:44.407763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d64270-2d69-4b5a-b488-333339187921 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:44.408166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d64270-2d69-4b5a-b488-333339187921 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:44.408311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d64270-2d69-4b5a-b488-333339187921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:44.427897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-66d64270-2d69-4b5a-b488-333339187921 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:27:44.429099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-66d64270-2d69-4b5a-b488-333339187921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:44.436907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 05:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:27:44.983273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:27:44.984927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:27:44.997751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534516af-ef57-4f4e-9660-e1444429aff5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:27:45.001255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:27:45.001871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:27:45.002148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:27:45.002434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:27:45.011161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:27:45.012567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f458028-5418-472b-9924-740da78bb462 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:27:45.022348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 780/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:27:54.422187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=11 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:27:55.018196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:09.212377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.213006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.213006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.237839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.239638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.254358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 760/1540] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:09.263101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71307e0-d537-4b04-937c-da2be692515c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.263647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71307e0-d537-4b04-937c-da2be692515c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.263647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71307e0-d537-4b04-937c-da2be692515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.299336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f71307e0-d537-4b04-937c-da2be692515c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.301701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f71307e0-d537-4b04-937c-da2be692515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.315066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 781/1541] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:09.320056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:09.336440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f94f0b-ce1c-4cf0-a293-c284c03aa745 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:09.343686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:09.344308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.344584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.344853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.345517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:09.345815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:09.348013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44d93-5430-4c4d-b478-1a5236956eee admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:09.369401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 761/1542] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:09.379183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71307e0-d537-4b04-937c-da2be692515c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:09.390047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71307e0-d537-4b04-937c-da2be692515c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:09.401513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:09.402141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.402437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.402724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.403380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:09.405295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:09.448293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d8f2-229d-4a62-944d-139f4fc1a8b4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.448567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d8f2-229d-4a62-944d-139f4fc1a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.449088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0d8f2-229d-4a62-944d-139f4fc1a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.484399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-40b0d8f2-229d-4a62-944d-139f4fc1a8b4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.485943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-40b0d8f2-229d-4a62-944d-139f4fc1a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.497111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 762/1543] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:09.507170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.511892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.511892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.537962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.539596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.555076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 763/1544] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:09.559309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:09.573134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75432a-53f3-4585-8888-c0822a5e8838 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:09.581246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:09.582192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.582729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.582870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 782/1545] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:09.583284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.584116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:09.584633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:09.590455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:09.592093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:09.606144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 764/1546] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:29:09.620459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:09.645029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a69e3-6c3d-4469-a40e-9776aeebbc31 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:09.736978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f0bf8c-a7e3-4872-a71f-40432e9bf5e4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:09.743569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:09.744397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.744683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.744953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.746528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:09.747613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:09.747613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:29:09.747613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:29:09.748793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f717e-d8ef-49e9-a840-6bd14c417fd6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:09.790116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:09.791126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.791672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.792156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.793219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:09.794516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:09.794516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:29:09.794516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:29:09.796349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7a043-1766-4416-bf62-08afddfc8368 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:09.868499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 783/1547] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/users => generated 284 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:09.870078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f24c-270f-4553-9b80-433101eabbba None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.870487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f24c-270f-4553-9b80-433101eabbba None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.870766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42f24c-270f-4553-9b80-433101eabbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.895377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7f42f24c-270f-4553-9b80-433101eabbba None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.899982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7f42f24c-270f-4553-9b80-433101eabbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.906196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 765/1548] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/projects => generated 379 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:09.906608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 784/1549] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:09.907594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.907919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.908194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.908311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:09.908505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:09.909003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:09.935037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.936309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:09.943144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:09.944355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.005668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 785/1550] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.006900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.018629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2e7bf-40c9-456a-9b62-5e7c87004138 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.024399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 766/1551] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.024399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.034005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.034670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.034987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.035259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.035959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.036841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325b24c-a753-46e6-a01f-b35c717cdc04 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.037564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52c937-76e5-4783-90f9-a53584e4fd65 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.041607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.042269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.043174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.043174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.043482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.045432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b3a85-f1ef-444b-b230-696265e96ae9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.050455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 786/1552] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:10.050455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1b0f0-0235-48f8-9644-efd1ed8f9727 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.050455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1b0f0-0235-48f8-9644-efd1ed8f9727 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.050759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1b0f0-0235-48f8-9644-efd1ed8f9727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.071385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d4e1b0f0-0235-48f8-9644-efd1ed8f9727 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.072588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d4e1b0f0-0235-48f8-9644-efd1ed8f9727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.080422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 787/1553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.081552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7e1f-28aa-422f-9324-3708be02ba74 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.082398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7e1f-28aa-422f-9324-3708be02ba74 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.088228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af7e1f-28aa-422f-9324-3708be02ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.101607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e2af7e1f-28aa-422f-9324-3708be02ba74 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.102807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e2af7e1f-28aa-422f-9324-3708be02ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.110755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 788/1554] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.114943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6a9c-bfff-4e2a-a98e-a6bd420d6fe2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.114943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6a9c-bfff-4e2a-a98e-a6bd420d6fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.114943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6a9c-bfff-4e2a-a98e-a6bd420d6fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.132578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-99fe6a9c-bfff-4e2a-a98e-a6bd420d6fe2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.133648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-99fe6a9c-bfff-4e2a-a98e-a6bd420d6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.140078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 767/1555] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:10.141608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215ac1-1976-48de-bcff-e31bca3c9662 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.141741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 789/1556] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.142178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215ac1-1976-48de-bcff-e31bca3c9662 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.142299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215ac1-1976-48de-bcff-e31bca3c9662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.143077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.143352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.143717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.162852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-60215ac1-1976-48de-bcff-e31bca3c9662 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.164064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-60215ac1-1976-48de-bcff-e31bca3c9662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.164254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.165422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.172129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 768/1557] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.173136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794306-fdf9-422c-a811-b044dae52156 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.173420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794306-fdf9-422c-a811-b044dae52156 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.177734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 790/1558] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.177885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd794306-fdf9-422c-a811-b044dae52156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.177983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.185601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa6be2-6051-42b9-b3b7-cf7841553392 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.190267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.190882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.191698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.191970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.192607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.194040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cd794306-fdf9-422c-a811-b044dae52156 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.194219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.195255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cd794306-fdf9-422c-a811-b044dae52156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.196522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:10.203111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 769/1559] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.204286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.204619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.204894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.205006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 791/1560] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:10.205769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.216939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3299a9-15a7-40e2-9c14-4147db335197 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.221633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.222250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.222896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.222896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.223504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.223810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.224822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:10.224822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:29:10.224822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:29:10.225709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e35562-39fe-4607-82d2-4c41a39bfe1b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.228392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.229591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.237720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 770/1561] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.239056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.249186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 792/1562] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.249186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.249434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:10.249520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.249520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.249696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f7b93-f6ce-4bbf-812c-f5858d46043c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.254891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.255508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.255871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.256083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.256729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.258355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.264726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.266065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.268648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 771/1563] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:10.269438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.274290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 793/1564] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.275605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.279082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073d176-a61d-4149-a659-9bd10203d603 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.284361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.284519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1fbeb-395c-45a9-8370-5f2aa0ebf189 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.285408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.285842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.286534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.287242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.287548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.289153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6670089-977e-47db-b328-be5b69993c06 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.290034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.290666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.294761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.294761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.294761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.294761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.294761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.301917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 772/1565] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.303258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.303540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.303813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.313067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 794/1566] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.314007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.324472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.327156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.327778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ca76d-afeb-4e88-8fd5-463510be8268 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.332521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.333204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.333480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.333729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 773/1567] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.333863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] PATH_INFO: `/v3/projects/88ba4bc214ac428c8319947c7bf7bd9b/users/acec3c130e7342cb8717d31e37bd803e/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.334547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.344564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a37188-efcc-449d-ba73-d7b5dba80059 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.349126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ba4bc214ac428c8319947c7bf7bd9b, user_id=acec3c130e7342cb8717d31e37bd803e, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.349453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.350047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.350326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.350624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.351259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.351588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.351821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.353286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.365460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 774/1568] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.366234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.375847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb57ff-047b-421d-b306-b44891380799 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.380916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.381506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.381782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.382051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.382902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.384774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.419316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 795/1569] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:10 2022] PUT /identity/v3/projects/88ba4bc214ac428c8319947c7bf7bd9b/users/acec3c130e7342cb8717d31e37bd803e/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:10.420207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.480626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b62a2c-2728-4243-ac03-fa589655b3f7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.484299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 775/1570] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:10.485245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.485391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.485850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.486128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.486399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.487066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.489633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.500054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 796/1571] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:10.500871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.511124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d3f88-57ab-4b94-9621-37ee1fc83ba4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.547792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b0657-86bb-42b0-819a-89a57825bd39 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.552966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.553779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.554058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.554330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.555671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.555980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.557606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.567412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.568019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.568299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.568564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.569198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.576787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 776/1572] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.576787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.577042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.580636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1487d8-fb33-4313-9efc-a0d6fa2b4e6e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.585785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.586386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.586693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.593827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.593827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.593827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.730669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 797/1573] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:10.731521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.756163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:10.763254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 777/1574] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/projects => generated 393 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:10.763254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.792798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199700bd-22ec-4a4e-be8c-8b8f893e747b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.822681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edff487-3f8a-478a-8d30-64cc352f8679 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.827551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.828298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.828579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.828847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.829475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.831519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.846106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:10.849425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.850033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.850312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.850614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.851262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:10.851563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.853150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e2fce-628d-43ea-8dcf-dad425aa7b20 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.864987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 798/1575] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:10.866162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.866467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.866749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.895604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 778/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:10.895604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.895894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] MFA Rules not processed for user `acec3c130e7342cb8717d31e37bd803e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:10.895894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:10.956281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7677d-6423-4aeb-94ff-1f5173854d66 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.959775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 799/1577] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:10.961021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:10.961597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.962212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.962521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.962795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.963427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:10.965365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:10.971890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871318d-3dc7-49e8-b537-ca4dc3ed80c6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:10.981337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:10.982087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:10.982535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:10.982824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] PATH_INFO: `/v3/projects/cc23f8d119514ad595637967b626e87d/users/55c9d3a3577746a4ad7406fe4396eaee/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:10.999196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc23f8d119514ad595637967b626e87d, user_id=55c9d3a3577746a4ad7406fe4396eaee, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.001861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.051422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 779/1578] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:10 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.052242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.061938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1092cf7-71ac-426e-9c56-d2c37bd348c6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.066554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.067132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.067407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.067677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.068287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.070181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.080813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 800/1579] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:10 2022] PUT /identity/v3/projects/cc23f8d119514ad595637967b626e87d/users/55c9d3a3577746a4ad7406fe4396eaee/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:11.082506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.115148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:11.145881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b393f5a-6995-4a00-91b5-74ba53e5b887 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.150641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.151247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.151524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.151793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.152412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.154317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.164371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 780/1580] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.166845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.228381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc9162-5f48-45b8-85dd-b22fd11397ae admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.233078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.233670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.233955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.234228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.234891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.236825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.240314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 801/1581] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.241328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.250670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af08e1-91c4-4330-a7ed-a12f8a010a55 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.255194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.255780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.256058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.256333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.256983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.259039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.334315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 781/1582] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.335510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.344832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532959a1-ba1c-4525-aa2b-1a54a6ae4d6f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.350060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.350658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.350932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.351200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.351803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.353711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.367034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 802/1583] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.367034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.420784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560670d9-ea4c-434c-b724-0794bd2060ff admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.424383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.424979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.425261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.425531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.452925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 782/1584] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.453764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.465099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392cd48-9212-4e5d-a781-9d83d149785a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.483931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.485387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc3a85-fdaf-4bb9-ae4e-d4c04f15379f service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.494560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 803/1585] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:11.495845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.496122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.496389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.505440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=11,keystone:SELECT=89 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:11.516251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:11.517697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:11.529563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.529868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.530452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.530760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.531713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.533769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.552417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 783/1586] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.552417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.556507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd5ad1-3c56-4415-a7c7-2845f2fa3079 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.561227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.561826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.562110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.562379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.563065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.565035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.570743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:11.587134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 804/1587] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:11.587912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.597557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd7020-f6fd-4cc8-b06f-41d5cad9ccfa None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.602720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.603324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.603606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.604072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.604621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.606225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.620335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 805/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.620335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.629006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249bac69-9586-430d-84ec-53e1879d16d4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.636871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.637600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.637956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.638340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.641743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.643905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.654945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 806/1589] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.655745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.665292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bf5bd-b1fa-4bee-8caa-2c1c89a9beac admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.670709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.671368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.671650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.671945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.672577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.674536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.677076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 784/1590] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.677244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.690342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f78fd5-23de-4c01-9996-a3a9fc6556cd admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.695810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.701499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.701499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.701499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.701499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.701499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.710283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 785/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.711185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.720768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b7ee1-328d-412c-9a16-b67099ff7f38 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.725481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.726118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.732605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.732605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.732605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.732605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.739929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 786/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.740717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.743700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:11.759963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 807/1593] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:11.759963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.770631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d1afae-e181-4385-ac23-00b165d981eb admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.776385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.777012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.777292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.778175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.779011 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.781766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.793633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 808/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.794615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.804636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6661bcc-3f43-4f84-97bf-1ffffd9e1d00 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.810230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.810852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.811139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.811584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.812249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.813621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da3e17-f0d7-4b58-8601-8df48efa0d98 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.813942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.818001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.818637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.818915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.819185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.824815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 809/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.825897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.828448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.829938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.837054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350380-402c-4a7d-8e32-b69b37dbbd05 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.839850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 787/1596] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:11.840576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.842820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375014-9f66-4841-8619-097863911463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.843651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375014-9f66-4841-8619-097863911463 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.844218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375014-9f66-4841-8619-097863911463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.844887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac375014-9f66-4841-8619-097863911463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.845748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375014-9f66-4841-8619-097863911463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.847439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac375014-9f66-4841-8619-097863911463 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.850625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c42deb-5198-4e70-b092-fd96d10d95db service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.856324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.856940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.865062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.865062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.865062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.865339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 810/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.865339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375014-9f66-4841-8619-097863911463 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.865503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.872237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac375014-9f66-4841-8619-097863911463 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.876507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 788/1598] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.876507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.880890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.881561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.881836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.882259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] PATH_INFO: `/v3/projects/a4880508c1824fa09919255293b90906/users/d357ef4db9054a32aff2daf2c5683b32/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.887805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d12217-f7c4-4819-8dca-506e356c2262 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.892602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.893205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.893504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.893763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.894466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.896118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.897526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4880508c1824fa09919255293b90906, user_id=d357ef4db9054a32aff2daf2c5683b32, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.900287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:11.906568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 789/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:11.907362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:11.916746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2bfda-49be-4ab7-99dc-b5606a4b82e2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:11.921340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:11.921955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:11.922238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:11.922551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] PATH_INFO: `/v3/projects/1c2ad5de95314180a9a4cf16c82bbe27/users/b69e39bc10c54335aa301ce2e98d8505/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:11.938092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2ad5de95314180a9a4cf16c82bbe27, user_id=b69e39bc10c54335aa301ce2e98d8505, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:11.939183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:11.939183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:29:11.939183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:29:11.941981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.007039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 811/1600] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:11 2022] PUT /identity/v3/projects/a4880508c1824fa09919255293b90906/users/d357ef4db9054a32aff2daf2c5683b32/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.007942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.034729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 790/1601] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:11 2022] PUT /identity/v3/projects/1c2ad5de95314180a9a4cf16c82bbe27/users/b69e39bc10c54335aa301ce2e98d8505/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.035576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.089559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122115-3348-4af3-8a3e-00255a877581 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.105192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:12.123403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cdb0c-4839-4264-8f37-2a4f4be0fe25 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.141600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.142964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.143247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.143698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.144392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.147287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.158009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.159191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.159524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.160531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] PATH_INFO: `/v3/projects/b34f5b5ce7c44930b3c85701d9ceb384/users/f4a01fadccd941fc98ecc99dac4e25c4/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.166039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 791/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:12.167297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.170346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:12.178649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81917-79c7-40ca-b01f-1d2a5fcb6caa admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.183822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.184728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.185082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b34f5b5ce7c44930b3c85701d9ceb384, user_id=f4a01fadccd941fc98ecc99dac4e25c4, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.185222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.185511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] PATH_INFO: `/v3/projects/1aff2725534245f0ae064361abeee90a/users/6e61341a0f5145499a3aeec91327a205/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.189355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.201352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aff2725534245f0ae064361abeee90a, user_id=6e61341a0f5145499a3aeec91327a205, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.205629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b585b-a815-4b6e-a925-3eede900dcb4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.251465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 812/1603] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:12 2022] PUT /identity/v3/projects/b34f5b5ce7c44930b3c85701d9ceb384/users/f4a01fadccd941fc98ecc99dac4e25c4/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.252375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.300091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 792/1604] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:12 2022] PUT /identity/v3/projects/1aff2725534245f0ae064361abeee90a/users/6e61341a0f5145499a3aeec91327a205/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.302001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.302294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.302733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.313624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79530ee-6c3d-4782-be3f-73312c6b4736 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.330399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] MFA Rules not processed for user `d357ef4db9054a32aff2daf2c5683b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.331809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.381778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.382358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.382637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.382914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.383564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.385356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d972e-08a0-4de1-a5ce-6b35d3efc2de admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.397868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 813/1605] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:12.399264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55480752-8c62-477c-b46b-8e2621daf75b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.400311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55480752-8c62-477c-b46b-8e2621daf75b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.400311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55480752-8c62-477c-b46b-8e2621daf75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.444821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-55480752-8c62-477c-b46b-8e2621daf75b None None] MFA Rules not processed for user `b69e39bc10c54335aa301ce2e98d8505`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.447756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-55480752-8c62-477c-b46b-8e2621daf75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.451387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 793/1606] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:12.452228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.495188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee786e18-e415-47d2-9358-8adb4782534f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.526118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.526591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.526972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.527677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] PATH_INFO: `/v3/projects/538589fdc80248a3a3e82802151b4eb6/users/f661527186794ba296b322c22a78f3f0/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.540936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 814/1607] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:12.542297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68907e-6b3a-47b3-93e2-6708340600fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.542612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68907e-6b3a-47b3-93e2-6708340600fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.542891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68907e-6b3a-47b3-93e2-6708340600fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.568901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9f68907e-6b3a-47b3-93e2-6708340600fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.571006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9f68907e-6b3a-47b3-93e2-6708340600fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.574285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538589fdc80248a3a3e82802151b4eb6, user_id=f661527186794ba296b322c22a78f3f0, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.577917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.585472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 815/1608] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:12.585472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284053f5-2588-43da-a221-40f3cf192102 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.585472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284053f5-2588-43da-a221-40f3cf192102 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.585472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284053f5-2588-43da-a221-40f3cf192102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.608478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:12.616536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-284053f5-2588-43da-a221-40f3cf192102 None None] MFA Rules not processed for user `6e61341a0f5145499a3aeec91327a205`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.618061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-284053f5-2588-43da-a221-40f3cf192102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.654084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 794/1609] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:12 2022] PUT /identity/v3/projects/538589fdc80248a3a3e82802151b4eb6/users/f661527186794ba296b322c22a78f3f0/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.655113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.714359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 816/1610] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:12.715998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98346-9c00-4a00-bda8-1b0a83aeb318 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.716353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98346-9c00-4a00-bda8-1b0a83aeb318 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.717193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c98346-9c00-4a00-bda8-1b0a83aeb318 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.718442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 817/1611] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 05:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:12.722168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45686f46-2825-48d6-af32-bfa77928aa51 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.722168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45686f46-2825-48d6-af32-bfa77928aa51 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.722168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45686f46-2825-48d6-af32-bfa77928aa51 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.722168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 818/1612] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 05:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:12.722591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45686f46-2825-48d6-af32-bfa77928aa51 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.741386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453d38b-0cb8-4aea-91bf-83b98a432da0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.746332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.747128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.747417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.747692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] PATH_INFO: `/v3/projects/1a200065c13e42babe6353689aabd896/users/11770fbaeacd4fc7bc4052e83630dbb3/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.763482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a200065c13e42babe6353689aabd896, user_id=11770fbaeacd4fc7bc4052e83630dbb3, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.766179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f06538-6a97-4e00-92af-eb7ca35ad561 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.786103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45686f46-2825-48d6-af32-bfa77928aa51 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.790027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.790768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.791124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.791471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.801256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.802731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b1fbe-eaa2-48ac-8408-22f8598e667b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:12.812060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 819/1613] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:12.813265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.813538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.813822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.832788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.834007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.841906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 820/1614] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:12.842686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:12.853216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 795/1615] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:12 2022] PUT /identity/v3/projects/1a200065c13e42babe6353689aabd896/users/11770fbaeacd4fc7bc4052e83630dbb3/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:12.854702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac13e41-a407-4002-ac77-67f1092137b3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:12.855656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81adb5-7b77-493e-b4db-0c4021064f3d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.859633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81adb5-7b77-493e-b4db-0c4021064f3d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.859633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81adb5-7b77-493e-b4db-0c4021064f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.870660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:12.903234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8b81adb5-7b77-493e-b4db-0c4021064f3d None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:12.904427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8b81adb5-7b77-493e-b4db-0c4021064f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:12.919566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:12.920475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:12.921091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:12.921524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:12.922391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:12.924772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee6adc-50ff-449b-8299-cb55c0ed0532 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.001280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 796/1616] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:13.002487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.002779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.003049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.022663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:13.023962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:13.033227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 797/1617] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:13.034085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.061649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 821/1618] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:12 2022] POST /identity/v3/projects => generated 403 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:13.061649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.061649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.061649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.079352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:13.080599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:13.087459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e709a-fe59-4e44-b833-8d16a6dfff89 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.145443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.145914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.146199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.149681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.155804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 822/1619] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:13.157315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.163235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.164647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.181124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 798/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:13.182008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.271115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cf9c2-5122-4fb1-898e-fd0761e8f868 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.276944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.277589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.277865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.278130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.295117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f940e5-9ed1-44d4-8276-1d25b0f854c7 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.301654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.302509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.302792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.303283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.370499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.372257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.377108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.378801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.384985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 823/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:13.386108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.394736 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 799/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:13.395754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.406265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d03f-63f4-442e-a371-1425cc1590d4 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.411572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.412881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.413333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.414012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.429124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.431233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.444373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:13.447512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 800/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:13.448749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215b9f5-4006-401b-b2b7-ae55b3122267 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.464008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.467886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.473110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.473110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.473110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.477760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.523628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:13.610962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 824/1624] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:13 2022] POST /identity/v3/users => generated 308 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:13.615138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.629892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649ee57-52be-4a7a-883e-f2e2659453c1 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.641561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.645916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.645916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.647239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.648216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.653248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa288a-eb4e-4fda-8f01-c8327ba13b5f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.677962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 801/1625] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:13.683811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.699419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d372c72-494d-48c7-9e25-ea146b62ba29 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.716146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.721992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.726192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.726192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] PATH_INFO: `/v3/projects/c885a2c2d4c7489f91bb023384372f28/users/7b702b32f8fc440b8dce23a008608c19/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.757075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48e31-0e9c-45ae-878f-c3cd2f624ad2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.757075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48e31-0e9c-45ae-878f-c3cd2f624ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.757386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b48e31-0e9c-45ae-878f-c3cd2f624ad2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.759115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 802/1626] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 05:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 05:29:13.763670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c885a2c2d4c7489f91bb023384372f28, user_id=7b702b32f8fc440b8dce23a008608c19, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.767802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.768197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.768621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.774063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53b1ae-3d6a-4f11-a547-10888e7a2e7b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.796259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:13.797706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:13.809857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 803/1627] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:29:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:13.824031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.874504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 825/1628] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:13 2022] PUT /identity/v3/projects/c885a2c2d4c7489f91bb023384372f28/users/7b702b32f8fc440b8dce23a008608c19/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:13.876083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3eb04-76c2-4289-93ae-29dc30328e01 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.876538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3eb04-76c2-4289-93ae-29dc30328e01 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.876965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3eb04-76c2-4289-93ae-29dc30328e01 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.881824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 826/1629] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 05:29:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:13.883118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3eb04-76c2-4289-93ae-29dc30328e01 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:13.929460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847823b3-9fc4-4848-8bbc-695818571310 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.962939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3eb04-76c2-4289-93ae-29dc30328e01 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:13.968794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.970160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.970160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.970347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:13.971255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:13.973624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3d5b2-4ec9-4f6f-871a-466a7231f0de admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:13.994813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:13.995483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:13.998488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:13.998488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:14.003121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 827/1630] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:14.007541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:14.007920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:14.008205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:14.055398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:14.056909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:14.093909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:14.117479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:14.121123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:14.155748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 804/1631] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:14.158048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 828/1632] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:29:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:14.159309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:14.177474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:14.184920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10bf2-5190-4173-a545-e6d7de254f2a service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:14.191263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b47eae-ecf2-4153-901f-53baeccf2021 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:14.195767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:14.198376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:14.198555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:14.198828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] PATH_INFO: `/v3/projects/c885a2c2d4c7489f91bb023384372f28/users/7b702b32f8fc440b8dce23a008608c19/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:14.199209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:14.200315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:14.200965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:14.201677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:14.202043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c885a2c2d4c7489f91bb023384372f28, user_id=7b702b32f8fc440b8dce23a008608c19, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:14.208183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc54d9a-5f83-4595-b7a3-7bfe26541770 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:14.217170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:14.220072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:14.221550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 805/1633] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:14 2022] PUT /identity/v3/projects/c885a2c2d4c7489f91bb023384372f28/users/7b702b32f8fc440b8dce23a008608c19/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:14.233305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:14.233709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:14.234655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:14.254821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 829/1634] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:14.304550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] MFA Rules not processed for user `7b702b32f8fc440b8dce23a008608c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:14.309019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:14.400418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:14.424618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 806/1635] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:14 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:15.717182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:15.717182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f1cce-3746-4098-b1d0-76440b96add6 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:15.717182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:15.717182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:15.720375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:15.720375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:15.720375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:15.720375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:15.720375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 830/1636] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:15 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:15.908593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:16.056850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:16.061826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719ca33-1cbd-4102-9a69-74974f1b6eb6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:16.071260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:16.072386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.072989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.073582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] PATH_INFO: `/v3/users/d357ef4db9054a32aff2daf2c5683b32` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.074872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661108cb-28b0-4f77-8572-51c1b43c6762 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:16.077377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d357ef4db9054a32aff2daf2c5683b32)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:16.078911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: WARNING py.warnings [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:16.078911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: warnings.warn(msg) Sep 01 05:29:16.078911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: Sep 01 05:29:16.082793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:16.092339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:16.093001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.093440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.093714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] PATH_INFO: `/v3/users/6e61341a0f5145499a3aeec91327a205` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.095674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e61341a0f5145499a3aeec91327a205)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:16.100673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:16.109659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:16.320299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:16.338100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e61341a0f5145499a3aeec91327a205'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.357275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd357ef4db9054a32aff2daf2c5683b32'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.362982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e61341a0f5145499a3aeec91327a205'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.381681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e61341a0f5145499a3aeec91327a205'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.382712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e61341a0f5145499a3aeec91327a205'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.383488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c35d057b-3fa4-4951-8858-1d7fa1bb9ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e61341a0f5145499a3aeec91327a205'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.391804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd357ef4db9054a32aff2daf2c5683b32'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.492939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 831/1637] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:16 2022] DELETE /identity/v3/users/6e61341a0f5145499a3aeec91327a205 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:16.498584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.499354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.499792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.523022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:16.524827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:16.558440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd357ef4db9054a32aff2daf2c5683b32'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.559199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd357ef4db9054a32aff2daf2c5683b32'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.598244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 832/1638] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:16.611115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:16.613316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:16.616659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-25dcd3f7-2dfa-41c4-b90f-29266182d28f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd357ef4db9054a32aff2daf2c5683b32'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:16.632628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 807/1639] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:15 2022] DELETE /identity/v3/users/d357ef4db9054a32aff2daf2c5683b32 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:16.636820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.637107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.637379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.674342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:16.676604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:16.739498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e33e77-0400-4871-87cb-b7d96e65a152 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:16.809593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:16.812118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.812727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.814388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.819804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:16.855519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 808/1640] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:16.894507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:16.896717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:16.939564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eaf6d8-84c8-4174-9b53-04e4c1519634 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:16.944561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:16.945425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:16.946068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:16.946540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:16.960613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:16.962696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:16.994830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 809/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:17.018577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:17.020277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:17.050789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 833/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:17.563197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:18.693388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:18.694897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:18.717181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c3c4a-4145-43bf-a0af-b558916c9f24 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:18.737359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:18.738286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:18.738784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:18.739279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] PATH_INFO: `/v3/projects/a4880508c1824fa09919255293b90906` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:18.744092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4880508c1824fa09919255293b90906)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:18.746168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:18.907793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:18.910562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:18.926671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeeaa00-644d-4f0b-88b6-d1fa9456d7ce service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:18.954133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4880508c1824fa09919255293b90906', 'request_id': 'req-80fc7390-0881-4e20-8c05-88d6df63c6a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:18.968049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4880508c1824fa09919255293b90906', 'request_id': 'req-80fc7390-0881-4e20-8c05-88d6df63c6a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.035695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:19.036737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.037236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.037728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] PATH_INFO: `/v3/users/b69e39bc10c54335aa301ce2e98d8505` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.039185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69e39bc10c54335aa301ce2e98d8505)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:19.046035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:19.060001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4880508c1824fa09919255293b90906', 'request_id': 'req-80fc7390-0881-4e20-8c05-88d6df63c6a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.081895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] The token cache is being invalidate because project a4880508c1824fa09919255293b90906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:19.082437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.083332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-80fc7390-0881-4e20-8c05-88d6df63c6a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.092988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 810/1643] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:18 2022] DELETE /identity/v3/projects/a4880508c1824fa09919255293b90906 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:19.098719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b3e49-db9f-4a8a-b071-2957fdee2429 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.099409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b3e49-db9f-4a8a-b071-2957fdee2429 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.099883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b3e49-db9f-4a8a-b071-2957fdee2429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.121387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:19.151144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-367b3e49-db9f-4a8a-b071-2957fdee2429 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:19.153351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-367b3e49-db9f-4a8a-b071-2957fdee2429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:19.180252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 811/1644] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:19.187889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed171a-f631-492c-99af-85a06853000d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.188172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed171a-f631-492c-99af-85a06853000d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.188441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed171a-f631-492c-99af-85a06853000d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.215349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e1ed171a-f631-492c-99af-85a06853000d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:19.216872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e1ed171a-f631-492c-99af-85a06853000d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:19.250746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 812/1645] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:19.251785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed171a-f631-492c-99af-85a06853000d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:19.265938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed171a-f631-492c-99af-85a06853000d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:19.271410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:19.271912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.274660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.275114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.275805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:19.276111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:19.278250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:19.300963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 813/1646] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:19.307171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:19.338971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2b7c4-05ed-4b48-9a08-7049790da270 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:19.360321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:19.360945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.361302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.361475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.362121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:19.365783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:19.371536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69e39bc10c54335aa301ce2e98d8505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.400680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69e39bc10c54335aa301ce2e98d8505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.429071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69e39bc10c54335aa301ce2e98d8505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.429071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69e39bc10c54335aa301ce2e98d8505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.429776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3019820d-29d0-4d2d-a360-f0969cc6402c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69e39bc10c54335aa301ce2e98d8505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:19.555073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 814/1647] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:19 2022] POST /identity/v3/projects => generated 405 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:19.561662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:19.563298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 834/1648] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:18 2022] DELETE /identity/v3/users/b69e39bc10c54335aa301ce2e98d8505 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:19.567320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:19.569755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.569974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.570253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.661486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:19.667403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:19.775335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:19.793925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bba2f2-2c06-44a0-8bce-33c6ad4f8276 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:19.843608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 835/1649] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:19.880703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:19.882332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:19.893014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:19.894618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:19.895092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:19.895698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:19.896830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:19.899969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.003851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 815/1650] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:19 2022] POST /identity/v3/users => generated 303 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:20.007767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.034952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444fdaab-649d-40eb-9474-a9256fa35c76 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.036419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffe83c-3f8d-420a-93a1-b32540635505 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.044468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.046716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.049162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.050511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.050833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.051313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.051536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.051643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.052184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.062572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.078321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.083147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.098069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 816/1651] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:20.099786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.112931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 836/1652] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:20.127828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99deefcb-f691-4074-b555-704e10c308c1 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.153499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.154128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.154419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.154844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.155510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.161992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.200998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 817/1653] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:20.205293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.216157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.219462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:20.222555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96e72b-a11d-4541-a6de-445c50290262 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.228316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.228989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.229316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.229681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] PATH_INFO: `/v3/projects/06f6ecc22a4c49c78b3de34b7ac31c52/users/85e7fa16d05840d3a7eb57c4f3a793a0/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.250276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14717181-4a7b-4e45-aa1d-157ac30d4fc2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.259901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f6ecc22a4c49c78b3de34b7ac31c52, user_id=85e7fa16d05840d3a7eb57c4f3a793a0, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.267074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.267838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.269193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.270070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] PATH_INFO: `/v3/projects/1aff2725534245f0ae064361abeee90a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.272655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aff2725534245f0ae064361abeee90a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.274037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ff1d0-3417-44ea-a575-c1490e714dcf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.280907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.381305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 837/1654] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:20 2022] PUT /identity/v3/projects/06f6ecc22a4c49c78b3de34b7ac31c52/users/85e7fa16d05840d3a7eb57c4f3a793a0/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:20.382925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.383210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.384623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.439481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:20.444511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:20.473298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aff2725534245f0ae064361abeee90a', 'request_id': 'req-81d4d65c-67ba-443d-b5bd-a1efceee43c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.484497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aff2725534245f0ae064361abeee90a', 'request_id': 'req-81d4d65c-67ba-443d-b5bd-a1efceee43c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.595855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aff2725534245f0ae064361abeee90a', 'request_id': 'req-81d4d65c-67ba-443d-b5bd-a1efceee43c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.621009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] The token cache is being invalidate because project 1aff2725534245f0ae064361abeee90a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:20.622878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.624794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.638875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 818/1655] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:20 2022] DELETE /identity/v3/projects/1aff2725534245f0ae064361abeee90a => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:20.642793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.648136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:20.656245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 838/1656] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:20.683269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4d65c-67ba-443d-b5bd-a1efceee43c4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.696690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.697764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.698505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.698951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] PATH_INFO: `/v3/users/7b702b32f8fc440b8dce23a008608c19` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:20.701458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b702b32f8fc440b8dce23a008608c19)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:20.704420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:20.812999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:20.814989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:20.828689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:20.960035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b702b32f8fc440b8dce23a008608c19'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.980310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d2eed0-40d6-46c0-b20c-1a86dbc0fde1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:20.981734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b702b32f8fc440b8dce23a008608c19'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:20.997012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:20.999120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:20.999460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:20.999819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:21.014672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:21.020474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:21.053575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 839/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:21.096190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b702b32f8fc440b8dce23a008608c19'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:21.096190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b702b32f8fc440b8dce23a008608c19'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:21.121609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-447cf965-0c20-4b14-b2cd-189bdd930555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b702b32f8fc440b8dce23a008608c19'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:21.135711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 819/1658] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:20 2022] DELETE /identity/v3/users/7b702b32f8fc440b8dce23a008608c19 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:21.197258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:21.513120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:21.519088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:21.547813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85918-75da-4866-88da-b32a131abc29 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:21.713195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:21.713957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:21.714307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:21.714692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] PATH_INFO: `/v3/projects/1c2ad5de95314180a9a4cf16c82bbe27` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:21.716662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2ad5de95314180a9a4cf16c82bbe27)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:21.717976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: WARNING py.warnings [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 05:29:21.717976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: warnings.warn(msg) Sep 01 05:29:21.717976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: Sep 01 05:29:21.724680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:22.036844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2ad5de95314180a9a4cf16c82bbe27', 'request_id': 'req-3fb07bca-2c59-470d-99c7-5737922016a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:22.157299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2ad5de95314180a9a4cf16c82bbe27', 'request_id': 'req-3fb07bca-2c59-470d-99c7-5737922016a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:22.165600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2ad5de95314180a9a4cf16c82bbe27', 'request_id': 'req-3fb07bca-2c59-470d-99c7-5737922016a5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:22.230036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] The token cache is being invalidate because project 1c2ad5de95314180a9a4cf16c82bbe27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:22.231350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:22.235750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3fb07bca-2c59-470d-99c7-5737922016a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:22.245177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 840/1659] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:21 2022] DELETE /identity/v3/projects/1c2ad5de95314180a9a4cf16c82bbe27 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:22.247469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed8bb-4d55-4b19-a826-c76cd19139b2 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.247918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed8bb-4d55-4b19-a826-c76cd19139b2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.248021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed8bb-4d55-4b19-a826-c76cd19139b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.290811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d33ed8bb-4d55-4b19-a826-c76cd19139b2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:22.292375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d33ed8bb-4d55-4b19-a826-c76cd19139b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:22.419547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 820/1660] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:22.427269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.427820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.428580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.433949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c50f-f175-4005-9be3-347e3a67a13c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.434060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c50f-f175-4005-9be3-347e3a67a13c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.434344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c50f-f175-4005-9be3-347e3a67a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.461866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:22.463192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:22.495499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 841/1661] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:22.506026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:22.514769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3cf3c50f-f175-4005-9be3-347e3a67a13c None None] MFA Rules not processed for user `55c9d3a3577746a4ad7406fe4396eaee`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:22.516974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3cf3c50f-f175-4005-9be3-347e3a67a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:22.538681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be44e8-1507-4f7a-bde7-811dd9673fc9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:22.558548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:22.559190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.559473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.559747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.560478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:22.560803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:22.562709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:22.605113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 842/1662] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:22.617429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:22.632804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956c2f-14ec-47e8-a993-4512f40f1492 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:22.642624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:22.644407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.644838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.645285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.646799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:22.650099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:22.690711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 821/1663] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:22.700780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.701607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.702089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.736657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] MFA Rules not processed for user `55c9d3a3577746a4ad7406fe4396eaee`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:22.738373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:22.746989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 822/1664] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:22.759445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:22.761173 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:22.777385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 843/1665] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:22.781599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:22.898211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:22.911104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091312ab-9a18-4795-b144-595a378e658c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:22.923170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:22.924188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:22.925450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:22.926061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:22.927005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:22.931308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:22.949940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbde2cc-c03c-4b92-bb78-36ce8b8069a3 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:22.979567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:23.070704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 844/1666] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:22 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:23.078607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.078607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.105604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302a7d4-ef17-4b70-b6bb-a618f0b435ad admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.111319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.111319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.111319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.113396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.135421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.136277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.136735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.138284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] PATH_INFO: `/v3/projects/c885a2c2d4c7489f91bb023384372f28` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.145546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c885a2c2d4c7489f91bb023384372f28)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.152946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.237062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.239300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.278503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 823/1667] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:23.279911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.307494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c885a2c2d4c7489f91bb023384372f28', 'request_id': 'req-0ada247a-648b-4d1d-99d7-d3eccfd3799a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:23.308018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f803d3-c7db-43a1-8d8f-4c28771f12fb service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.378185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c885a2c2d4c7489f91bb023384372f28', 'request_id': 'req-0ada247a-648b-4d1d-99d7-d3eccfd3799a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:23.387435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c885a2c2d4c7489f91bb023384372f28', 'request_id': 'req-0ada247a-648b-4d1d-99d7-d3eccfd3799a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:23.421793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] The token cache is being invalidate because project c885a2c2d4c7489f91bb023384372f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:23.422088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:23.422851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0ada247a-648b-4d1d-99d7-d3eccfd3799a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:23.424520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 845/1668] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:23 2022] DELETE /identity/v3/projects/c885a2c2d4c7489f91bb023384372f28 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:23.428863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.429840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.431128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.431561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.431825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621776-163c-46eb-a8c5-61171215b802 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.432281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621776-163c-46eb-a8c5-61171215b802 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.432567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e621776-163c-46eb-a8c5-61171215b802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.434846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.439430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.479213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7e621776-163c-46eb-a8c5-61171215b802 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:23.480643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7e621776-163c-46eb-a8c5-61171215b802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.483578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 824/1669] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:23.484692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.487565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.503243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 846/1670] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:23.512515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.513007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.513424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.528261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fd01b-2179-456e-b108-bf2e78ea9f9a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.543300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.544104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.544722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.545141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.546177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.548024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.571165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:23.575925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.576717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 825/1671] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:23.578702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.597030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4805b5-3e70-4793-9215-a42557834ec0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.600565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 847/1672] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:23.601537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.608418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.609557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.610050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.610604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] PATH_INFO: `/v3/projects/b21b8ad53adb4778b07a6ac63ffcec59/users/ce8bd24dde8441ab89fcf34eed164fbe/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.617581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7e26-39cd-42e7-8c72-baf5cdee2de8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.628795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.633611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.633934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.634211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.639849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21b8ad53adb4778b07a6ac63ffcec59, user_id=ce8bd24dde8441ab89fcf34eed164fbe, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.641733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:23.642060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.644421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbfa461-f026-491a-a164-f847ce056cdd admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.645270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.671021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 848/1673] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:29:23.674599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:23.691780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd945cc-3b78-4dcf-b552-6747465d0e25 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:23.708439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:23.710286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.710965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.712058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.712896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:23.716431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d3aa5-306b-4f44-b865-9e3fd8908071 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:23.752058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 826/1674] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:23 2022] PUT /identity/v3/projects/b21b8ad53adb4778b07a6ac63ffcec59/users/ce8bd24dde8441ab89fcf34eed164fbe/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:23.765803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.766102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.766377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.809098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] MFA Rules not processed for user `ce8bd24dde8441ab89fcf34eed164fbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:23.810530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.866845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 849/1675] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:23.868276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:23.868919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:23.869803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:23.922110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] MFA Rules not processed for user `85e7fa16d05840d3a7eb57c4f3a793a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:23.924021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:23.938768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 827/1676] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:23.939645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.024487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 850/1677] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:24.029690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.101049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af34d-9435-4a01-b19f-9ea59f3d6074 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.241479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbdeeb-5a91-491e-a133-29de9ad08af3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.255525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.257068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.257357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.259211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.259879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.278803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.298549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.302466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.302838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.303124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.303938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.309190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.432175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 851/1678] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:24 2022] POST /identity/v3/projects => generated 379 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:24.432654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 828/1679] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:23 2022] POST /identity/v3/users => generated 282 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:24.432654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.440735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:24.440895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.467009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d32f44-45fa-4493-af83-21a6edba521b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.587962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.588660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.589009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.589351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.590060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.593060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27360c-3964-404e-b952-e78da6e6330f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.627698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf71415-37c2-4e7c-a092-f826fea1f028 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.633962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.634628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.635001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.635281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.637182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.639401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.655103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 852/1680] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:24.656146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.668847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9557f-0a44-42c7-be4b-3803aca25d3f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.677879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.679258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.680044 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.689233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.689233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.689233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.730385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 829/1681] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:24 2022] POST /identity/v3/projects => generated 403 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:24.731864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1190-109f-47ab-b29a-09d2a94f97f0 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.732158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1190-109f-47ab-b29a-09d2a94f97f0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.746965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e1190-109f-47ab-b29a-09d2a94f97f0 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.746965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 830/1682] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 05:29:24 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:24.749076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e1190-109f-47ab-b29a-09d2a94f97f0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.861046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:24.870272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 853/1683] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:24 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:24.872084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:24.905321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce5bd0e-5482-4986-ba56-3d4da16fa0c1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.914000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e1190-109f-47ab-b29a-09d2a94f97f0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:24.922062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.922750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.923033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.923305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.929048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.929184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:24.929184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.929184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.929402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e40a99-3acc-45c7-89f8-b88a0a8892fe admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.936699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.937734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:24.944533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 831/1684] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:24.945206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:24.948008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:24.948647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:24.957815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:24.958612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:25.045296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:25.047114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:25.071264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 854/1685] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:24 2022] POST /identity/v3/users => generated 308 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:25.072512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.094641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c25c48-fa60-40a8-98a1-fb73d6d9ea2c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.107707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.108649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.109140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.109555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.110458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.116185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.143644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 855/1686] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:25.144652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.150247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 832/1687] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:29:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:25.151683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.169901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930794db-7645-44fd-86cd-0a2033cd14ee admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.171214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b6db8-090d-4c9f-9f52-6da06224743e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.178284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.180085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.180794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.181202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.182171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.184238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.185203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.186013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.186297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.186602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] PATH_INFO: `/v3/projects/e873f7beb8b34e15a098ef3abbb3f6c0/users/349dfb2270874d9798f42a42efc952b1/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.212468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 833/1688] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:25.213662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.228015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e873f7beb8b34e15a098ef3abbb3f6c0, user_id=349dfb2270874d9798f42a42efc952b1, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.233836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.240857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf4670-4565-4452-b222-8fbc4056602f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.256517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.257440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.257629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.257908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.258643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.262574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.276006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 834/1689] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:25.277945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.280127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:25.304911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93321135-15ae-48c6-b6be-f6bd5b29ecb8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.304911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.305630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.305630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.305820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.317110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.326179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.328816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 856/1690] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:25 2022] PUT /identity/v3/projects/e873f7beb8b34e15a098ef3abbb3f6c0/users/349dfb2270874d9798f42a42efc952b1/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:25.340444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.346928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 835/1691] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:25.347848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.375723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5d2ec-a398-474c-83f8-8a0e3f2854dd service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.442264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.443724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.443724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.443944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] PATH_INFO: `/v3/projects/d275eef8e57047948f251984e54cafca/users/5260c8ca57b54d78bcbb6e613b07db0e/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.473277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f969-03ad-45d5-9852-0d17b55e5ed2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.474733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:25.482482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275eef8e57047948f251984e54cafca, user_id=5260c8ca57b54d78bcbb6e613b07db0e, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.487309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.488891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.489207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b14dc1-d5a6-4447-a213-19a17141d61a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.489647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.491411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] PATH_INFO: `/v3/projects/59169fc8df7e4380b13b3811fbfd5b65/users/d5a851a4592a499dad36fae1357f5233/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.512973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59169fc8df7e4380b13b3811fbfd5b65, user_id=d5a851a4592a499dad36fae1357f5233, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.516078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.586130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 836/1692] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:25 2022] PUT /identity/v3/projects/d275eef8e57047948f251984e54cafca/users/5260c8ca57b54d78bcbb6e613b07db0e/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:25.594138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22efa3-63d8-4b64-a62d-6d1db1c816a0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.594611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22efa3-63d8-4b64-a62d-6d1db1c816a0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.594908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22efa3-63d8-4b64-a62d-6d1db1c816a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.611061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 857/1693] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:25 2022] PUT /identity/v3/projects/59169fc8df7e4380b13b3811fbfd5b65/users/d5a851a4592a499dad36fae1357f5233/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:25.613369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:25.618039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:25.678395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9c22efa3-63d8-4b64-a62d-6d1db1c816a0 None None] MFA Rules not processed for user `349dfb2270874d9798f42a42efc952b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:25.679929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9c22efa3-63d8-4b64-a62d-6d1db1c816a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:25.790319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c0e06-6997-4094-9048-eb6e91fb8b8d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:25.804460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:25.807187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.807187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.807187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] PATH_INFO: `/v3/users/acec3c130e7342cb8717d31e37bd803e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.807725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=acec3c130e7342cb8717d31e37bd803e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:25.812802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:25.871079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 837/1694] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:25.872717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:25.873129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:25.873499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:25.926609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] MFA Rules not processed for user `5260c8ca57b54d78bcbb6e613b07db0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:25.928534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:26.101346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acec3c130e7342cb8717d31e37bd803e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:26.104798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 838/1695] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:26.105926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.122869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e1bdd-ff88-460d-b97f-2f8afdef1a48 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.132775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:26.139994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.140727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.142214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.144445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:26.147749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acec3c130e7342cb8717d31e37bd803e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:26.149157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:26.169672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acec3c130e7342cb8717d31e37bd803e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:26.171400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acec3c130e7342cb8717d31e37bd803e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:26.172568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acec3c130e7342cb8717d31e37bd803e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:26.175387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 839/1696] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:26.178607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.185619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:26.217174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6f8be-a0b3-4c31-8a16-484066d9fcd2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.226598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:26.227232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.227613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.227889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.228539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:26.230899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:26.323656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 858/1697] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:25 2022] DELETE /identity/v3/users/acec3c130e7342cb8717d31e37bd803e => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:26.324579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.325994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:26.371349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 840/1698] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:26 2022] POST /identity/v3/projects => generated 365 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:26.372549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.378275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:26.468190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7f5b9-69dd-4039-9e5e-18aafb1026cd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.551651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f451e-26ba-4761-8169-265de8bcecd6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.571223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:26.571404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:26.572679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.573609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.573985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.574256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.575166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:26.576335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.576722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] PATH_INFO: `/v3/projects/59169fc8df7e4380b13b3811fbfd5b65/users/d5a851a4592a499dad36fae1357f5233/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.578546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:26.583307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59169fc8df7e4380b13b3811fbfd5b65, user_id=d5a851a4592a499dad36fae1357f5233, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:26.592731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd2418-bb78-4319-924e-874d8c1fd01e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:26.617556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 841/1699] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:26 2022] PUT /identity/v3/projects/59169fc8df7e4380b13b3811fbfd5b65/users/d5a851a4592a499dad36fae1357f5233/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:26.620079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.624370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.624370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.652798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:26.652798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:26.713416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 859/1700] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:26 2022] POST /identity/v3/projects => generated 379 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:26.714660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.731978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:26.804893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 842/1701] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:26.806126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:26.811556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:26.851676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76db8a-5d63-4965-af63-b2d471b36f8b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.893106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2d749-b031-44f1-a637-7b59f2fb8cab admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:26.912641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:26.914176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:26.917446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:26.917446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:26.917446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:26.919735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.007566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.008383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.008761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.009128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.009927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.012927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.034730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 860/1702] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:26 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:27.035874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.038045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:27.147107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 843/1703] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:26 2022] POST /identity/v3/users => generated 296 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:27.148203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.171277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6693b78-58dc-4c26-9d99-d78fedfd4223 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.187974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.199711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.200155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.200553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.201379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.203177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.229509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203cec8-9de9-4d90-9eeb-a26c6ecc7fc5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.247395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 844/1704] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:27.249648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.251408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.259583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:27.259900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.260471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.261477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.264960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:27.306943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.317212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.350279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 861/1705] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:27.352954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.381268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c43db8-2786-4e24-878f-656d4b0785bc service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.387335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611f5b-d023-4306-b43a-2536b005e6a9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.396216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.397090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.397943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.399502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.409627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.410243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.411092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.411485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.411856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.416793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.437546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.440433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.449321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 862/1706] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:27.453175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.466845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 845/1707] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:27.467619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.476081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f6b29-9910-4a3e-b6e1-ddc7bfe3fb27 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.485416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770062b-01c7-4077-a3fb-6c1c6b2e5faf service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.493931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.496607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.497052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.497391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.503078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] PATH_INFO: `/v3/projects/e86101a452b94542b49ad85a61579495/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.503222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.503222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.503222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] PATH_INFO: `/v3/projects/fdd91fec87844eabaab75214b442dd6d/users/119d5edf373246f2bd1da9250e419d7e/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.542080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd91fec87844eabaab75214b442dd6d, user_id=119d5edf373246f2bd1da9250e419d7e, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.545089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.550350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86101a452b94542b49ad85a61579495, user_id=eaaefb2fc9e9489cbe0a2b3d8f39e67c, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.558999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:27.568450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:27.662657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 846/1708] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:27 2022] PUT /identity/v3/projects/fdd91fec87844eabaab75214b442dd6d/users/119d5edf373246f2bd1da9250e419d7e/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:27.666636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.707375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:27.719555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 863/1709] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:27 2022] PUT /identity/v3/projects/e86101a452b94542b49ad85a61579495/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:27.723676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:27.856684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f3e4-7b29-489b-80f4-1855f722fc46 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.937273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdcc16-76c8-4e28-b90c-d51f23762046 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:27.950764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:27.951577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:27.951862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:27.952139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:27.953748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:27.955426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.015587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 864/1710] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:28.026065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:28.090371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9b0ca-b335-412b-bed3-cfefa655525c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:28.098067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.098421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.102197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.102197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.102197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:28.102197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.109475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.112838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.113258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.113734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] PATH_INFO: `/v3/projects/e86101a452b94542b49ad85a61579495/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.119728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86101a452b94542b49ad85a61579495, user_id=eaaefb2fc9e9489cbe0a2b3d8f39e67c, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:28.131858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0ad6a-4881-487f-8c80-aa4ee7056581 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.135515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 847/1711] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:28.139591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:28.155491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11edbc1-9aa7-4b7e-b25b-e3be7e12c88f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:28.172126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.172838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.175296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.175296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] PATH_INFO: `/v3/projects/fdd91fec87844eabaab75214b442dd6d/users/119d5edf373246f2bd1da9250e419d7e/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.180493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd91fec87844eabaab75214b442dd6d, user_id=119d5edf373246f2bd1da9250e419d7e, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:28.197235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 865/1712] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:28 2022] PUT /identity/v3/projects/e86101a452b94542b49ad85a61579495/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:28.198268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73503b-4aca-4f63-8fc6-8dcc4dcd9ead admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.199139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.199432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.199712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.238628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 848/1713] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:28 2022] PUT /identity/v3/projects/fdd91fec87844eabaab75214b442dd6d/users/119d5edf373246f2bd1da9250e419d7e/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:28.240446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.240738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.241015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.279215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] MFA Rules not processed for user `eaaefb2fc9e9489cbe0a2b3d8f39e67c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:28.287008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.370913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] MFA Rules not processed for user `119d5edf373246f2bd1da9250e419d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:28.374622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.478764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 866/1714] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:28.491907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:28.493894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.626280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 849/1715] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:28.642572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:28.645331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.670518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4344f1-cf9a-4820-8799-454a0065cca1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:28.690229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e4c25-274d-4a2d-8cc0-2b5f8097e73c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:28.691917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.692621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.692846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.693123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.699338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.702110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.702628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.703905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.727340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:28.729012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad50e2-362f-467c-a95f-90ef9d4af23e service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.755412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 850/1716] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:28.757144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:28.758371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:28.797562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 867/1717] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:28.850125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.850692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.850692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.906048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] MFA Rules not processed for user `f661527186794ba296b322c22a78f3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:28.911014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.934130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:28.951938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425818b9-e53b-4bb7-b769-44a1759d60ec service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:28.961904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:28.963620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:28.965796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:28.966464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:28.970878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:28.983588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:29.090080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 851/1718] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:29.112255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:29.113833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:29.139436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:29.141848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:29.176013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 868/1719] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:29.178252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:29.201776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5db6a-b779-420f-94ca-9b39319664c1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:29.221491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:29.221491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:29.221491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:29.221491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:29.223890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:29.311799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3b073-afff-4ec4-9fbc-57b582da9b6c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:29.320824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:29.322621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:29.322621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:29.322621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:29.332424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:29.334455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:29.346548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 852/1720] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:29.393467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:29.398115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497422f-febc-4e6e-802d-eafe3beadf60 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:29.418740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:29.432414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 869/1721] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:30.153962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c192b-ca55-41e1-a272-19a5e1790e28 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:30.154565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c192b-ca55-41e1-a272-19a5e1790e28 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:30.154645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:30.155117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c192b-ca55-41e1-a272-19a5e1790e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:30.157398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:30.248519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bf7c192b-ca55-41e1-a272-19a5e1790e28 None None] MFA Rules not processed for user `d5a851a4592a499dad36fae1357f5233`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:30.249868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bf7c192b-ca55-41e1-a272-19a5e1790e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:30.272833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3f16d-9c1a-4cfb-af05-612f7bfe68fc service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:30.280430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:30.281603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:30.281890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:30.286973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] PATH_INFO: `/v3/projects/88ba4bc214ac428c8319947c7bf7bd9b` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:30.286973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ba4bc214ac428c8319947c7bf7bd9b)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:30.286973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:30.407805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 870/1722] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:30.478090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ba4bc214ac428c8319947c7bf7bd9b', 'request_id': 'req-bb3b3894-44e2-4b4b-9334-6cfd891795f1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:30.520357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ba4bc214ac428c8319947c7bf7bd9b', 'request_id': 'req-bb3b3894-44e2-4b4b-9334-6cfd891795f1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:30.646147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ba4bc214ac428c8319947c7bf7bd9b', 'request_id': 'req-bb3b3894-44e2-4b4b-9334-6cfd891795f1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:30.673051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] The token cache is being invalidate because project 88ba4bc214ac428c8319947c7bf7bd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:30.673355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:30.678950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bb3b3894-44e2-4b4b-9334-6cfd891795f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:30.681764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 853/1723] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:30 2022] DELETE /identity/v3/projects/88ba4bc214ac428c8319947c7bf7bd9b => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:30.686584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:30.687203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:30.687669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:30.750610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:30.760632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:30.933788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 871/1724] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:30.941544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:30.941834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:30.942135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:30.998225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:31.001278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:31.001971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:31.035825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 854/1725] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:31.037355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.049971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8062264-aa06-44d5-9dba-fdb2a8cf3c41 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.056510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.057130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.057407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.057677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.058335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:31.059989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:31.061686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:31.129670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 872/1726] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:31.132739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.136585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:31.150449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2aeeb-b662-4598-98f9-27963dabf7e5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.189814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.190462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.190902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.191363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.191922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:31.193861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef385d56-3080-433a-a7d0-2c7902f7536f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:31.402680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 855/1727] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:31 2022] POST /identity/v3/projects => generated 373 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:31.407464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.420023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a870d3-8e91-4767-9d94-6ea93a9045d7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.434943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c54f-42cb-4c81-b102-4050893ed67a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.435460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c54f-42cb-4c81-b102-4050893ed67a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.435928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043c54f-42cb-4c81-b102-4050893ed67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.507943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d043c54f-42cb-4c81-b102-4050893ed67a None None] MFA Rules not processed for user `f4a01fadccd941fc98ecc99dac4e25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:31.509005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d043c54f-42cb-4c81-b102-4050893ed67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:31.591909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.605685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.610661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.610661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.610661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:31.610661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:31.642308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 856/1728] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:31.652987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.653563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.653846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.712516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] MFA Rules not processed for user `f4a01fadccd941fc98ecc99dac4e25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:31.713789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:31.736925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 857/1729] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:31.748920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.750465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:31.815055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 873/1730] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:31 2022] POST /identity/v3/users => generated 287 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:31.823704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.841348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e69001-d60b-4ef9-b09e-25a30d882430 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.867712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.868604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.870770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.874633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.875463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:31.902534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:31.931544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398e13e-32a3-4659-b3cc-f78884b5ea43 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.944965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 874/1731] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:31.946912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.948405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:31.949054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.949414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.949826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.972117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:31.973969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:31.976366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c36c37-3bde-4ba0-b87e-9db2527ad15c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:31.996856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:31.997822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:31.998301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:31.998847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:31.999719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:32.001562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc5d1c-52ca-48c2-b9eb-16b4c01c5b78 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:32.011108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 858/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:32.031595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 875/1733] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:32.032220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:32.067523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06963009-6845-4c68-b378-f8596a09a3b3 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:32.081420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:32.082107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:32.082393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:32.082707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] PATH_INFO: `/v3/projects/ea7ae437f48041f08164153e7a4945ef/users/58ff9c1d5b4b44b1815fc9252df81867/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:32.143691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7ae437f48041f08164153e7a4945ef, user_id=58ff9c1d5b4b44b1815fc9252df81867, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:32.146768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:32.285858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 859/1734] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:32 2022] PUT /identity/v3/projects/ea7ae437f48041f08164153e7a4945ef/users/58ff9c1d5b4b44b1815fc9252df81867/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:32.297676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce89af-4f95-407c-984a-b626a1f922c4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:32.298261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce89af-4f95-407c-984a-b626a1f922c4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:32.298656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce89af-4f95-407c-984a-b626a1f922c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:32.373186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-03ce89af-4f95-407c-984a-b626a1f922c4 None None] MFA Rules not processed for user `58ff9c1d5b4b44b1815fc9252df81867`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:32.375662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-03ce89af-4f95-407c-984a-b626a1f922c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:32.522160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:32.541997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 876/1735] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:32.546547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:32.624582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81479e8-89f6-49be-afc0-9ec5d5ee0671 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:32.624870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81479e8-89f6-49be-afc0-9ec5d5ee0671 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:32.625169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81479e8-89f6-49be-afc0-9ec5d5ee0671 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:32.626718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 877/1736] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:32.645075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:32.737530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577085f6-6ad1-44f8-9f47-12579df2b60d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:32.755343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:32.763984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:32.764296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:32.764577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:32.766009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:32.774638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:32.840099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:32.840390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:32.840723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:32.887745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:32.892231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:32.897541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 860/1737] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:32 2022] POST /identity/v3/projects => generated 371 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:32.909418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:33.125185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 878/1738] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:29:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:33.136505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:33.142145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50090e94-2176-403b-9cbe-b29355012269 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:33.149617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:33.150914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.151625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.152183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.153496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:33.157208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:33.219347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:33.221427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:33.254303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 861/1739] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:32 2022] POST /identity/v3/users => generated 292 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:33.257639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:33.273957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c14140-fb45-473a-bb40-262cdc51ddee None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:33.297040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a6c46-46fd-4388-83df-b0d82ad247e1 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:33.299983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:33.302622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.302913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.303181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.333055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:33.340919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.340919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.342620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.343743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:33.351229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:33.351907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:33.358033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a54d9a-8f56-4866-a738-be032a1c416f service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:33.381402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 862/1740] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:33.383374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:33.394373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 879/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:33.437947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25412ac1-a672-48eb-a9e0-935b7ec23ec2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:33.462868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:33.465017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.465454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.469196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] PATH_INFO: `/v3/projects/db6e2ab5ef324a2990eea647107fe6f2/users/f1555a881b76470984ee3a7203e92f15/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.474570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abf27-d803-4deb-95fc-f0a20facd0a2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.475047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abf27-d803-4deb-95fc-f0a20facd0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.475468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5abf27-d803-4deb-95fc-f0a20facd0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.507347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6e2ab5ef324a2990eea647107fe6f2, user_id=f1555a881b76470984ee3a7203e92f15, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:33.510205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:33.590103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3f5abf27-d803-4deb-95fc-f0a20facd0a2 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:33.592648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3f5abf27-d803-4deb-95fc-f0a20facd0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:33.669129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 863/1742] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:33 2022] PUT /identity/v3/projects/db6e2ab5ef324a2990eea647107fe6f2/users/f1555a881b76470984ee3a7203e92f15/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:33.670605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:33.771676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:33.791553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 880/1743] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:33.792946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adacb-4f46-48cd-818b-bf87e253156f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.793233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adacb-4f46-48cd-818b-bf87e253156f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.793507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295adacb-4f46-48cd-818b-bf87e253156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.876671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-295adacb-4f46-48cd-818b-bf87e253156f None None] MFA Rules not processed for user `f4a01fadccd941fc98ecc99dac4e25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:33.882354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-295adacb-4f46-48cd-818b-bf87e253156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:33.938776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac156f-5abc-4a3b-9ffd-be5016468b5d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:33.958401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:33.959310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:33.959585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:33.959858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:33.960507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:33.963157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:34.007309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 864/1744] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:34.008085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 881/1745] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:34.010067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:34.041034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5ce-88b7-4c17-bed3-1c22ac010f24 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:34.072265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:34.074748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:34.077640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:34.078473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] PATH_INFO: `/v3/projects/db6e2ab5ef324a2990eea647107fe6f2/users/f1555a881b76470984ee3a7203e92f15/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:34.096963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6e2ab5ef324a2990eea647107fe6f2, user_id=f1555a881b76470984ee3a7203e92f15, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:34.099911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:34.129176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 865/1746] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:34 2022] PUT /identity/v3/projects/db6e2ab5ef324a2990eea647107fe6f2/users/f1555a881b76470984ee3a7203e92f15/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:34.139169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8aa5-025b-4648-b026-9d8b0d1c83e8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:34.139466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8aa5-025b-4648-b026-9d8b0d1c83e8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:34.139719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b8aa5-025b-4648-b026-9d8b0d1c83e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:34.241493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f07b8aa5-025b-4648-b026-9d8b0d1c83e8 None None] MFA Rules not processed for user `f1555a881b76470984ee3a7203e92f15`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:34.242540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f07b8aa5-025b-4648-b026-9d8b0d1c83e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:34.378269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 882/1747] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:34.390799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:34.393967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:34.492561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2ddf1-a8d9-4eb5-a8d8-cda35712ab7a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:34.497284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:34.498104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:34.499468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:34.500060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:34.521602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:34.523543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3939e37-7a8d-4133-a0d8-bc9716ec8fde service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:34.542725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 866/1748] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:35.015346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f784dc-9fd0-49bc-b9fa-53782262b97e None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.015848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f784dc-9fd0-49bc-b9fa-53782262b97e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.015952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f784dc-9fd0-49bc-b9fa-53782262b97e None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.017549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 883/1749] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 05:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:35.026237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7c98-711d-455d-b54b-226211f39fc6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.026788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7c98-711d-455d-b54b-226211f39fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.027323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f7c98-711d-455d-b54b-226211f39fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.052640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.053156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.053580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.053852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-237f7c98-711d-455d-b54b-226211f39fc6 None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:35.055320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-237f7c98-711d-455d-b54b-226211f39fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:35.100628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] MFA Rules not processed for user `11770fbaeacd4fc7bc4052e83630dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:35.102127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:35.170726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 867/1750] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:35.172309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae5657-15b2-4657-a3ad-f6469e9a25d2 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.172477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae5657-15b2-4657-a3ad-f6469e9a25d2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.172660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae5657-15b2-4657-a3ad-f6469e9a25d2 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.174018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 868/1751] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:35.181361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc860-398a-4f7f-8f7f-20e1d1580639 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.181653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc860-398a-4f7f-8f7f-20e1d1580639 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.181933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfc860-398a-4f7f-8f7f-20e1d1580639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.213342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8ebfc860-398a-4f7f-8f7f-20e1d1580639 None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:35.215087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8ebfc860-398a-4f7f-8f7f-20e1d1580639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:35.226015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 884/1752] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:35.226015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:35.230158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:35.248274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c554d0-8c6f-4a3a-99a5-4d9d0e4b66e8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:35.252996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:35.253909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.254194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.254491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.264459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:35.266442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:35.278636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 885/1753] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:35.280353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:35.314985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 869/1754] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:35.326914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de25d3-c66c-4760-98a5-139b73a28632 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.327202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de25d3-c66c-4760-98a5-139b73a28632 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.327479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de25d3-c66c-4760-98a5-139b73a28632 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.333625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 870/1755] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 1 05:29:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:35.334987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.335270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.335547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.374094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] MFA Rules not processed for user `0c57f8b0816a4832b24043daea93e113`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:35.374731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304e514-01c5-4245-84ab-cb90664c3dbb service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:35.375370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:35.386644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 871/1756] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 1 05:29:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:35.388801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:35.389497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.389854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.390202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.395244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:35.406842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:35.410619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:35.415207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d1a81-3651-4508-8a9f-c407f50ce3f6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:35.424872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:35.425897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:35.426368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:35.426856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:35.432482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 886/1757] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:35.444027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:35.446104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:35.478155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 872/1758] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:36.119015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:36.119545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:36.121321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:36.122209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:36.459864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09646e26-ff4b-42b0-a456-2ca0e1054b1c service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:36.471520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:36.472401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:36.472873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:36.473490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] PATH_INFO: `/v3/users/349dfb2270874d9798f42a42efc952b1` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:36.475309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] RBAC: Authorizing `identity:delete_user(user_id=349dfb2270874d9798f42a42efc952b1)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:36.477633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:36.515794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa4d3-eed0-47c3-866f-51a0a6190e71 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:36.539195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:36.540070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:36.540967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:36.546882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:36.734738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:36.737346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:36.750570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:36.754765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 887/1759] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:29:36.759363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:36.785905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0647-aaab-45bb-a8ed-acfcb5de42ce service swift] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:36.795617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:36.799864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:36.800540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:36.800660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349dfb2270874d9798f42a42efc952b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:36.801120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:36.810808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349dfb2270874d9798f42a42efc952b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:36.958700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:36.960271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:36.980020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 888/1760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:36.998932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349dfb2270874d9798f42a42efc952b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:36.999619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349dfb2270874d9798f42a42efc952b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.003200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:37.012367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:37.029393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349dfb2270874d9798f42a42efc952b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.052351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 873/1761] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:36 2022] DELETE /identity/v3/users/349dfb2270874d9798f42a42efc952b1 => generated 0 bytes in 931 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:37.052351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:37.058233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:37.066119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e4d8a-2ebb-4569-8f57-c094ff42a938 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:37.283650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:37.284466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:37.284604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:37.292408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] PATH_INFO: `/v3/users/ce8bd24dde8441ab89fcf34eed164fbe` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:37.299000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8bd24dde8441ab89fcf34eed164fbe)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:37.310551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:37.317104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9787b0-5f25-4b3d-b04a-932fc49da68b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:37.326460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:37.327711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:37.328261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:37.328461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] PATH_INFO: `/v3/users/85e7fa16d05840d3a7eb57c4f3a793a0` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:37.331626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e7fa16d05840d3a7eb57c4f3a793a0)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:37.336463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:37.490442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:37.666245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8bd24dde8441ab89fcf34eed164fbe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.681627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e7fa16d05840d3a7eb57c4f3a793a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.698880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e7fa16d05840d3a7eb57c4f3a793a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.723040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8bd24dde8441ab89fcf34eed164fbe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.740629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8bd24dde8441ab89fcf34eed164fbe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.749924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8bd24dde8441ab89fcf34eed164fbe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.750703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-28e735c7-5f44-4a98-b4bc-d61efad70264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8bd24dde8441ab89fcf34eed164fbe'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.808737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:37.879168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e7fa16d05840d3a7eb57c4f3a793a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.880148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e7fa16d05840d3a7eb57c4f3a793a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.919757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c424c3dd-ad05-4a14-853d-d6340bd72b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e7fa16d05840d3a7eb57c4f3a793a0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:37.944196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 874/1762] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:37 2022] DELETE /identity/v3/users/85e7fa16d05840d3a7eb57c4f3a793a0 => generated 0 bytes in 887 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:37.944196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c714ec-b018-41ec-8c0d-3956147c3a4d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:37.944196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c714ec-b018-41ec-8c0d-3956147c3a4d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:37.944196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c714ec-b018-41ec-8c0d-3956147c3a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:37.970110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 889/1763] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:36 2022] DELETE /identity/v3/users/ce8bd24dde8441ab89fcf34eed164fbe => generated 0 bytes in 990 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:37.971805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:37.972223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:37.972640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.007392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-82c714ec-b018-41ec-8c0d-3956147c3a4d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:38.009879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-82c714ec-b018-41ec-8c0d-3956147c3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.013574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:38.014870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.133492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 875/1764] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:38.133492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.133492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.133492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.149022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 890/1765] 10.222.0.60 () {58 vars in 1015 bytes} [Thu Sep 1 05:29:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:38.159437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:38.159437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.168581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 876/1766] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:38.259764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:38.266433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.271137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:38.288554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775600e4-8e3f-4302-8ed7-189ad751b5bc None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:38.297567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95c8b9-8578-414f-89af-51d4f7ab8396 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:38.298199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:38.302367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.303518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.308249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:38.308592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.310937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.311562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.311994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.340214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:38.362088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:38.363230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:38.363230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31a594-8ca5-499c-9705-b89e75c26941 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:38.363564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 877/1767] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:38.363564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:38.363564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.371898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 891/1768] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:38.374486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd781-24de-4caf-b560-edae059740f9 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:38.379374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:38.380136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.380510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.380877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.389694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:38.391315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:38.402175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 878/1769] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:38.662174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.662780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.663265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.713602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] MFA Rules not processed for user `f661527186794ba296b322c22a78f3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:38.715675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.854348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 892/1770] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:38.904078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:38.906676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:38.914749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:38.940315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe797fde-3a98-4355-9000-ea172d2d6773 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:38.957470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:38.958063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:38.966494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:38.966494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] PATH_INFO: `/v3/projects/06f6ecc22a4c49c78b3de34b7ac31c52` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:38.966494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f6ecc22a4c49c78b3de34b7ac31c52)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:38.976113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:39.235990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f6ecc22a4c49c78b3de34b7ac31c52', 'request_id': 'req-eb51f3d6-a201-47b5-b085-5702edbbeb6c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.293524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f6ecc22a4c49c78b3de34b7ac31c52', 'request_id': 'req-eb51f3d6-a201-47b5-b085-5702edbbeb6c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.390268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f6ecc22a4c49c78b3de34b7ac31c52', 'request_id': 'req-eb51f3d6-a201-47b5-b085-5702edbbeb6c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.414884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] The token cache is being invalidate because project 06f6ecc22a4c49c78b3de34b7ac31c52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:39.415204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.419641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.422467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 879/1771] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:38 2022] DELETE /identity/v3/projects/06f6ecc22a4c49c78b3de34b7ac31c52 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:39.426824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:39.466660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76ea57-b089-46cf-bfda-5d6df2cb5ace None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:39.578546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:39.652686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:39.653534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:39.653833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:39.654110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] PATH_INFO: `/v3/users/d5a851a4592a499dad36fae1357f5233` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:39.664782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a851a4592a499dad36fae1357f5233)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:39.667055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:39.896122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:39.958879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a851a4592a499dad36fae1357f5233'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:39.999304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a851a4592a499dad36fae1357f5233'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.020481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a851a4592a499dad36fae1357f5233'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.022941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a851a4592a499dad36fae1357f5233'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.024686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a851a4592a499dad36fae1357f5233'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.163252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 893/1772] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:39 2022] DELETE /identity/v3/users/d5a851a4592a499dad36fae1357f5233 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:40.221651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:40.223894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:40.258950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51f3d6-a201-47b5-b085-5702edbbeb6c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:40.291558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:40.297034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:40.407001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:40.408107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:40.410795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:40.411411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] PATH_INFO: `/v3/projects/b21b8ad53adb4778b07a6ac63ffcec59` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:40.414396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21b8ad53adb4778b07a6ac63ffcec59)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:40.418460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:40.616068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a14f3-51e5-46da-83ae-c4e6c59f0c71 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:40.619422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21b8ad53adb4778b07a6ac63ffcec59', 'request_id': 'req-6dfe4388-ca85-48f5-a386-9c35376a3d22'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.646635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21b8ad53adb4778b07a6ac63ffcec59', 'request_id': 'req-6dfe4388-ca85-48f5-a386-9c35376a3d22'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.751031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21b8ad53adb4778b07a6ac63ffcec59', 'request_id': 'req-6dfe4388-ca85-48f5-a386-9c35376a3d22'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.787622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] The token cache is being invalidate because project b21b8ad53adb4778b07a6ac63ffcec59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:40.789979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.793119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:40.806802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 880/1773] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:40 2022] DELETE /identity/v3/projects/b21b8ad53adb4778b07a6ac63ffcec59 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:40.808018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:40.809530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:40.822820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:40.823995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:40.824446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:40.824943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] PATH_INFO: `/v3/projects/e873f7beb8b34e15a098ef3abbb3f6c0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:40.849383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe4388-ca85-48f5-a386-9c35376a3d22 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:40.850843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e873f7beb8b34e15a098ef3abbb3f6c0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:40.855919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:40.862062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:40.863067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:40.865825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:40.868519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] PATH_INFO: `/v3/users/5260c8ca57b54d78bcbb6e613b07db0e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:40.871605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5260c8ca57b54d78bcbb6e613b07db0e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:40.885164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:41.182311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e873f7beb8b34e15a098ef3abbb3f6c0', 'request_id': 'req-37f688e3-30dc-4859-b579-d3b181745b3f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.198339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5260c8ca57b54d78bcbb6e613b07db0e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.224062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5260c8ca57b54d78bcbb6e613b07db0e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.255220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e873f7beb8b34e15a098ef3abbb3f6c0', 'request_id': 'req-37f688e3-30dc-4859-b579-d3b181745b3f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.276214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e873f7beb8b34e15a098ef3abbb3f6c0', 'request_id': 'req-37f688e3-30dc-4859-b579-d3b181745b3f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.330632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] The token cache is being invalidate because project e873f7beb8b34e15a098ef3abbb3f6c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:41.330632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.331500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.335736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 894/1774] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:40 2022] DELETE /identity/v3/projects/e873f7beb8b34e15a098ef3abbb3f6c0 => generated 0 bytes in 1043 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:41.338336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:41.339802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:41.361211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5260c8ca57b54d78bcbb6e613b07db0e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.362799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:41.363655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5260c8ca57b54d78bcbb6e613b07db0e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.414973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5366199c-1908-4dbb-ac48-4d47202488be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5260c8ca57b54d78bcbb6e613b07db0e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:41.467108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 881/1775] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:40 2022] DELETE /identity/v3/users/5260c8ca57b54d78bcbb6e613b07db0e => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:41.520090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:41.734589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f688e3-30dc-4859-b579-d3b181745b3f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:41.992224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:41.992879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:41.998886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:41.999439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] PATH_INFO: `/v3/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:42.001326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaaefb2fc9e9489cbe0a2b3d8f39e67c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:42.004325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:42.334358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaaefb2fc9e9489cbe0a2b3d8f39e67c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:42.362211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaaefb2fc9e9489cbe0a2b3d8f39e67c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:42.377360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaaefb2fc9e9489cbe0a2b3d8f39e67c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:42.378161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaaefb2fc9e9489cbe0a2b3d8f39e67c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:42.379162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3aed7106-9278-4c6c-b83a-b04f1755625e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaaefb2fc9e9489cbe0a2b3d8f39e67c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:42.516515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 895/1776] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:41 2022] DELETE /identity/v3/users/eaaefb2fc9e9489cbe0a2b3d8f39e67c => generated 0 bytes in 1168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:42.654690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:42.655628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:42.656887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:42.662208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 882/1777] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:29:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:42.719661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:42.721129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:42.721471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:42.756160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:42.758279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:42.849099 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 896/1778] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:29:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:42.867634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:42.871920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:42.910738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f812c-9ad0-496c-b84f-71dc83f78992 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:42.937244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:42.938096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:42.943462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:42.943971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:42.982057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:43.049664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637c70e-9e24-4b3a-866a-408b5617b905 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:43.049664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 883/1779] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:43.432898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:43.434508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:43.557868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:43.654363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4bd57-93f4-425d-9e7c-63673c346af6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:43.659209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:43.676652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:43.676864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:43.678851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] PATH_INFO: `/v3/projects/59169fc8df7e4380b13b3811fbfd5b65` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:43.681048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=59169fc8df7e4380b13b3811fbfd5b65)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:43.684107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:43.701377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:43.853428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59169fc8df7e4380b13b3811fbfd5b65', 'request_id': 'req-b766510d-12b5-4976-81c6-ef1097aeb6cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:43.995485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59169fc8df7e4380b13b3811fbfd5b65', 'request_id': 'req-b766510d-12b5-4976-81c6-ef1097aeb6cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.005659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59169fc8df7e4380b13b3811fbfd5b65', 'request_id': 'req-b766510d-12b5-4976-81c6-ef1097aeb6cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.023459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] The token cache is being invalidate because project 59169fc8df7e4380b13b3811fbfd5b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:44.033947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.033947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b766510d-12b5-4976-81c6-ef1097aeb6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.036229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 897/1780] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:43 2022] DELETE /identity/v3/projects/59169fc8df7e4380b13b3811fbfd5b65 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:44.041350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.042931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.043209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.088113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:44.091194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.230551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 884/1781] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:44.232470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.233465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.239227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.312343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:44.313624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.327071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 898/1782] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:44.328405 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.343060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c71bde-f5b0-41a0-a722-8f7926d2a341 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.353069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.353580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.354063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.354671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.355497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:44.356639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.358893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.374891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.379360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.389257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 885/1783] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:44.390683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.395952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c783a-d8dd-4305-8309-56d08ae1fa12 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.402667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.403659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.404921 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06da4-4789-47f4-b28a-729ab1f41214 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.405073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.405500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] PATH_INFO: `/v3/projects/e86101a452b94542b49ad85a61579495` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.406883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86101a452b94542b49ad85a61579495)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.410343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.425487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.427500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.429777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.430245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.431720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.437335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.555299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86101a452b94542b49ad85a61579495', 'request_id': 'req-0d67e014-1d1d-4374-899c-2b6aee925d0f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.585725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 886/1784] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:44.588995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.668363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159864b-a31f-480f-9772-be85fe80840d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.678162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.678847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.679128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.679398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.680793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.683855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86101a452b94542b49ad85a61579495', 'request_id': 'req-0d67e014-1d1d-4374-899c-2b6aee925d0f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.685507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc92c2e-102e-4e89-8353-12604850ffd7 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.702569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86101a452b94542b49ad85a61579495', 'request_id': 'req-0d67e014-1d1d-4374-899c-2b6aee925d0f'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.722047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] The token cache is being invalidate because project e86101a452b94542b49ad85a61579495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:44.722570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.723578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:44.733183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 899/1785] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:44 2022] DELETE /identity/v3/projects/e86101a452b94542b49ad85a61579495 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:44.734722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.737702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.808759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 887/1786] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:44.808759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.808759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.808759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.832149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:44.833545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67e014-1d1d-4374-899c-2b6aee925d0f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.833791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.838459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.839098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.839377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.839682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] PATH_INFO: `/v3/projects/d275eef8e57047948f251984e54cafca` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.840786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d275eef8e57047948f251984e54cafca)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.842712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.849747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 888/1787] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:44.849747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.849747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.858957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5127904-1805-4c01-96cb-bdef3c820073 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.866148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.866867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.867217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.867562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.868284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.869972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82ccb9-dd1e-41ba-8482-48ab89520754 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.883017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 889/1788] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:44.884197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.884576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.885006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.906217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:44.907588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:44.915756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 890/1789] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:44.916556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.926735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daee09-1346-41de-9fca-97d2dd399c49 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.931546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:44.934992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:44.934992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:44.934992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:44.934992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:44.934992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:44.949319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 891/1790] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:44.949319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:44.975623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281af68e-f8b6-4d77-b8d8-1dfd99218d09 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:44.984561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd275eef8e57047948f251984e54cafca', 'request_id': 'req-b6703190-7dab-4888-804c-afbd9dc8bf4d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.059854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.062945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.063438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.065666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.066897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:45.067440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.070949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.081270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd275eef8e57047948f251984e54cafca', 'request_id': 'req-b6703190-7dab-4888-804c-afbd9dc8bf4d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.086883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 892/1791] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:45.088093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.092598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd275eef8e57047948f251984e54cafca', 'request_id': 'req-b6703190-7dab-4888-804c-afbd9dc8bf4d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.099574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b35d5-62c5-4f58-9fd4-5525da8fbcb3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.105598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.111231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.111231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.111231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] PATH_INFO: `/v3/projects/9a996409ea3e4d39828895a711a40a9e/users/e48cd79fbd5047ea930d5aea116853bc/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.111508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] The token cache is being invalidate because project d275eef8e57047948f251984e54cafca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:45.111770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.112527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.114438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 900/1792] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:44 2022] DELETE /identity/v3/projects/d275eef8e57047948f251984e54cafca => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:45.115394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.117043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.126869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a996409ea3e4d39828895a711a40a9e, user_id=e48cd79fbd5047ea930d5aea116853bc, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.129775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.142500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703190-7dab-4888-804c-afbd9dc8bf4d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.150003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.150863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.151264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.151611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.162698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.162698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44981329-dda2-42f9-ad0b-bc62243675bc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.225089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 893/1793] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:45 2022] PUT /identity/v3/projects/9a996409ea3e4d39828895a711a40a9e/users/e48cd79fbd5047ea930d5aea116853bc/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:45.228346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.228346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.260336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 901/1794] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:45.261549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfc235-2257-4026-847a-82d87d050a25 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.261968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfc235-2257-4026-847a-82d87d050a25 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.262256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfc235-2257-4026-847a-82d87d050a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.283186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-aecfc235-2257-4026-847a-82d87d050a25 None None] MFA Rules not processed for user `11770fbaeacd4fc7bc4052e83630dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:45.284448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-aecfc235-2257-4026-847a-82d87d050a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.298756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1cfd5-970e-499e-ada1-1f77980de46d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.346199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 902/1795] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:45.350864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.350864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.350864 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.360326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.361028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.361305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.361580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] PATH_INFO: `/v3/users/119d5edf373246f2bd1da9250e419d7e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.362749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=119d5edf373246f2bd1da9250e419d7e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.364749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.378804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] MFA Rules not processed for user `e48cd79fbd5047ea930d5aea116853bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:45.380329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.451512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 903/1796] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:45.452576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.472165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155b0a4-e954-436a-bb28-b3e1f2e2066f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.478331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.478991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.479276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.486102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.486889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.489046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.598808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '119d5edf373246f2bd1da9250e419d7e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.608328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '119d5edf373246f2bd1da9250e419d7e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.618049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 904/1797] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:45.619133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.623695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.643228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9691001-0660-4076-94e1-d34336f2ee57 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.650863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.651902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.653541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.654040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.655137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.658098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:45.738204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '119d5edf373246f2bd1da9250e419d7e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.738996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '119d5edf373246f2bd1da9250e419d7e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.776037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 905/1798] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:45.777918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.780664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '119d5edf373246f2bd1da9250e419d7e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:45.796621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 894/1799] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:45 2022] DELETE /identity/v3/users/119d5edf373246f2bd1da9250e419d7e => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:45.797772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:45.800471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:45.932067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df4d5e-33e9-41e5-9ffa-db60eebcbdac admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.962289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb8407-a7d9-4bd0-a43b-cca8341b7935 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:45.975353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:45.976257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:45.976531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:45.976804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:45.977473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:45.979453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.058383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.059345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.059726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.060178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.060940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.062744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.076840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 906/1800] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:46.083315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.085601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:46.100628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 895/1801] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:45 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:46.102555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa3460f-d0d7-453a-9e70-1221d1f959ed admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.104782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.114676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.115469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.115760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.116130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.116512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:46.117036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.120122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28468385-e084-4e22-a44a-7ddec0062264 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.120966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.125725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.127052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.127455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.128222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.129042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.131106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.139238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 907/1802] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:46.144390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 896/1803] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:46.148066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.148389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.161359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bfdac3-8db9-4d3c-a36d-8f2dfc37382b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.168239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c128b-4db1-4202-a43b-33f23f4193c9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.169721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.170916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.171391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.171848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] PATH_INFO: `/v3/projects/d2272721bf874bcd839a865884f523c8/users/f256f881dd9045b0a9cb3638a7e84258/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.181353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:46.192224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.193246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.194028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.195877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] PATH_INFO: `/v3/projects/6010c6661ac54fc185be5dd21827876b/users/cecba3cdd6324cd6944e82dc82ee6e18/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.198619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2272721bf874bcd839a865884f523c8, user_id=f256f881dd9045b0a9cb3638a7e84258, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.202355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.229384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6010c6661ac54fc185be5dd21827876b, user_id=cecba3cdd6324cd6944e82dc82ee6e18, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.235516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce3a48-ca8c-4e20-8974-72a84e1ba304 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.312431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 897/1804] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:46 2022] PUT /identity/v3/projects/d2272721bf874bcd839a865884f523c8/users/f256f881dd9045b0a9cb3638a7e84258/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:46.317147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.350550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 908/1805] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:46 2022] PUT /identity/v3/projects/6010c6661ac54fc185be5dd21827876b/users/cecba3cdd6324cd6944e82dc82ee6e18/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:46.354811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.355096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.355367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.417052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632b2f0-1c10-431c-a8a6-c376a4a6b709 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.483991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] MFA Rules not processed for user `cecba3cdd6324cd6944e82dc82ee6e18`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:46.501158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:46.563464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.564570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.567082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.567082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.567082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.571601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.611942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 898/1806] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:46.612700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.645967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac527b-7112-406e-8e07-8fd55b1c00fb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:46.655162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:46.656463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.656999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.658026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] PATH_INFO: `/v3/projects/d2272721bf874bcd839a865884f523c8/users/f256f881dd9045b0a9cb3638a7e84258/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.660666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2272721bf874bcd839a865884f523c8, user_id=f256f881dd9045b0a9cb3638a7e84258, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:46.673223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec594111-e6a5-4287-b125-3dc5c0b890e1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:46.687643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 899/1807] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:46 2022] PUT /identity/v3/projects/d2272721bf874bcd839a865884f523c8/users/f256f881dd9045b0a9cb3638a7e84258/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:46.705568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:46.706000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:46.707081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:46.767804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 909/1808] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:46.770106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:46.773793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:46.803170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] MFA Rules not processed for user `f256f881dd9045b0a9cb3638a7e84258`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:46.804779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:46.809330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976dce08-fd80-4e17-a9f3-5aecc75af2b0 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:47.038745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:47.041524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.041949 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.042234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] PATH_INFO: `/v3/projects/fdd91fec87844eabaab75214b442dd6d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.065845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd91fec87844eabaab75214b442dd6d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:47.075244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:47.122732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 900/1809] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:47.125750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:47.174002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708ce1-debc-43e7-bda2-4df87ff5a3d5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:47.195394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:47.195394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.195394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.195394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.195717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:47.195717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:47.352113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 901/1810] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:47 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:47.354089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:47.356604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:47.380595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd91fec87844eabaab75214b442dd6d', 'request_id': 'req-15512568-a2f9-4152-bcd6-f7882ea6abec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:47.460836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd91fec87844eabaab75214b442dd6d', 'request_id': 'req-15512568-a2f9-4152-bcd6-f7882ea6abec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:47.469633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd91fec87844eabaab75214b442dd6d', 'request_id': 'req-15512568-a2f9-4152-bcd6-f7882ea6abec'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:47.550610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] The token cache is being invalidate because project fdd91fec87844eabaab75214b442dd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:47.550911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:47.551613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:47.553227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 910/1811] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:46 2022] DELETE /identity/v3/projects/fdd91fec87844eabaab75214b442dd6d => generated 0 bytes in 785 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:47.554089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:47.554796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:47.558582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:47.601013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:47.602741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074458-5b04-4801-9b3a-796377d01115 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:47.617646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:47.622497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.622793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.623589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.648481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15512568-a2f9-4152-bcd6-f7882ea6abec admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:47.648705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:47.719337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:47.721825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.721952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.722288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.725619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:47.729901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6554c-471c-46b4-985a-5f37cb73247d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:47.852808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 911/1812] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:47 2022] POST /identity/v3/users => generated 290 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:47.854273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.856178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.856178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.882982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:47.888204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:47.889655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:47.897619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558229a-5cb2-4de3-8a16-8d196b7248e1 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:47.904550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 912/1813] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:47.905535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:47.918434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 902/1814] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:47.919888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.920754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3698-3de6-40fa-8399-3bca0c363857 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:47.922097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.922388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.927078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:47.927542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:47.927829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:47.928103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:47.929473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:47.931582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:47.966303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 913/1815] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:47.967964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:47.978996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:47.980873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:47.995271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 903/1816] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:48.000344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.013965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63f647-a094-46c3-945d-d519a05ea79f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.020963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b8d8f-fd1b-4dbf-b9d2-88c9173eaed4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.028249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.029406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.029406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.029406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] PATH_INFO: `/v3/projects/23d9adcd82494cc894bd900ac97e07e2/users/9ea725380bf44fbaadbe7095db441613/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.044999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.046009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.046553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.047030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.048083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:48.048621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.050510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.064976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d9adcd82494cc894bd900ac97e07e2, user_id=9ea725380bf44fbaadbe7095db441613, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.067415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.075972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 904/1817] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:29:48.080325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.115409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9400e-8265-4ad6-b607-1f96833981da admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.127896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.128583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.128822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.129067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.132967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.135569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.231669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 914/1818] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:47 2022] PUT /identity/v3/projects/23d9adcd82494cc894bd900ac97e07e2/users/9ea725380bf44fbaadbe7095db441613/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:48.233674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.375653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56ce49-2de0-441f-b130-41eeaaf049e9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.394263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 905/1819] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:48 2022] POST /identity/v3/projects => generated 383 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:48.401696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.402027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.406220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.406948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.410786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.411891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.419461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.464056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 915/1820] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:48.464056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.464056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:48.523129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731285dc-7d82-48b9-a47d-2a2caa284742 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.527863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3aa79f-a517-4fff-b7e0-7865f78ca29e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.554760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.555506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.555927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.557036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.559109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.561310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.713414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.714184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.714607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.714947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.728599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:48.958953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.958953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.958953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 916/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:48.958953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:48.958953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9063-1c22-49dd-8bc6-45c9c8e8ea48 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:48.960778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:48.960778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.960778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.960778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] PATH_INFO: `/v3/projects/23d9adcd82494cc894bd900ac97e07e2/users/9ea725380bf44fbaadbe7095db441613/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d9adcd82494cc894bd900ac97e07e2, user_id=9ea725380bf44fbaadbe7095db441613, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da9fb0-392e-42f7-a8a6-39b9754d5cc2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 917/1822] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:48 2022] PUT /identity/v3/projects/23d9adcd82494cc894bd900ac97e07e2/users/9ea725380bf44fbaadbe7095db441613/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1134f9-6e21-4bae-b2c1-cdbe6795de81 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1134f9-6e21-4bae-b2c1-cdbe6795de81 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.961473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1134f9-6e21-4bae-b2c1-cdbe6795de81 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:48.962114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 918/1823] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:48.962114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff573d3-99c5-43c1-9bfc-c434aad92e6e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:48.962114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff573d3-99c5-43c1-9bfc-c434aad92e6e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:48.962114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff573d3-99c5-43c1-9bfc-c434aad92e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.004822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-eff573d3-99c5-43c1-9bfc-c434aad92e6e None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:49.006751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eff573d3-99c5-43c1-9bfc-c434aad92e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:49.171295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 919/1824] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:29:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:49.171295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.171295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.171295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.210935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] MFA Rules not processed for user `9ea725380bf44fbaadbe7095db441613`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:49.217265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:49.260199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 906/1825] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:48 2022] POST /identity/v3/users => generated 292 bytes in 859 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:49.261841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:49.272283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:49.394042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 920/1826] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:49.400145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:49.496667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7fefd-498a-4ace-af4a-508912588de0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:49.506356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:49.507355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.507735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.508119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.522724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:49.526163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:49.547554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 907/1827] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:49.551412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:49.555187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:49.591301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5673ee8-0ccc-44da-98c8-b50e5dd02a3d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:49.629023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:49.629804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.631882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.633258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.634061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:49.643172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:49.703275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 921/1828] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:49.706658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:49.737680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f72673-b65b-4aac-888a-810f1b9901d6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:49.745040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f21b45-3990-42ba-b437-ca59a783c662 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:49.758621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:49.758621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.758621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.758621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.759558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:49.759558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:49.785500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 922/1829] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:49.790075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:49.791298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:49.797450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.799123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.799539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.810794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298734a5-5347-4e47-be48-b36445d5855b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:49.826716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:49.827351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:49.827629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:49.827901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] PATH_INFO: `/v3/projects/c70e17dd32a14444b1d3d27f985d8691/users/44b6eaccde2044f595163115504213cd/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:49.836307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:49.841936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9600ca2-68f9-4821-aee8-74fb44d74289 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:49.882286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 908/1830] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:49.893282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70e17dd32a14444b1d3d27f985d8691, user_id=44b6eaccde2044f595163115504213cd, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:49.893282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:50.015383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 923/1831] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:49 2022] PUT /identity/v3/projects/c70e17dd32a14444b1d3d27f985d8691/users/44b6eaccde2044f595163115504213cd/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:50.026958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:50.027505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:50.028047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:50.157902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] MFA Rules not processed for user `44b6eaccde2044f595163115504213cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:50.165410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:50.398704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 909/1832] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:50.471188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:50.512791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:50.623383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:50.698875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97903fbc-871e-409c-811e-ef0912f4b3aa admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:50.703305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:50.704918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:50.705453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:50.705923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:50.717761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:50.719573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee22aa-632b-4be8-b044-0f3321bb5601 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:50.742253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 924/1833] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:50.782211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:50.926487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6691-3081-4625-9986-12a69f7a71ae None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:50.934307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:50.940887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:50.940887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:50.940887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] PATH_INFO: `/v3/users/44b6eaccde2044f595163115504213cd` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:50.940887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b6eaccde2044f595163115504213cd)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:50.940887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:51.212989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b6eaccde2044f595163115504213cd'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:51.231185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b6eaccde2044f595163115504213cd'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:51.383923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b6eaccde2044f595163115504213cd'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:51.397434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b6eaccde2044f595163115504213cd'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:51.433597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b6eaccde2044f595163115504213cd'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:51.464727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 910/1834] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:50 2022] DELETE /identity/v3/users/44b6eaccde2044f595163115504213cd => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:51.469477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:51.469770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:51.470051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:51.528377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:51.539492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:51.642026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 925/1835] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:51.695662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:51.697512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:51.900251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052c740-4df4-451f-9af2-7d4a3694fb7c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:51.920498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:51.921872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:51.921872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:51.921872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:51.938038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:51.942566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c1575-3bf1-4c8d-a049-d311bc371af3 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:51.973954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 911/1836] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:53.056007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.060022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:53.082582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76219-c05b-482f-8ff0-ce48501fa170 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.091788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.094172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.094425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.094701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] PATH_INFO: `/v3/projects/c70e17dd32a14444b1d3d27f985d8691` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.096136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70e17dd32a14444b1d3d27f985d8691)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.098187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.238582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70e17dd32a14444b1d3d27f985d8691', 'request_id': 'req-09fc5b90-10f7-4c73-adab-037c1f0a300c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:53.344173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70e17dd32a14444b1d3d27f985d8691', 'request_id': 'req-09fc5b90-10f7-4c73-adab-037c1f0a300c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:53.351202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70e17dd32a14444b1d3d27f985d8691', 'request_id': 'req-09fc5b90-10f7-4c73-adab-037c1f0a300c'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:53.365744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] The token cache is being invalidate because project c70e17dd32a14444b1d3d27f985d8691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:53.366219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:53.366927 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-09fc5b90-10f7-4c73-adab-037c1f0a300c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:53.368894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 926/1837] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:53 2022] DELETE /identity/v3/projects/c70e17dd32a14444b1d3d27f985d8691 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:53.373856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.378509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.378509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.394648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:53.395906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:53.486553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 912/1838] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:53.494353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.494657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.495097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.516389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:53.517742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:53.526678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 927/1839] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:53.530369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.546765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb8c74-b579-4904-9936-bfb58f3442a2 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.553636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.554073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.554363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.554676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.555341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:53.555644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.557328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.572201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 913/1840] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:53.576406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.586570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d407a-a843-4169-8b17-8ef66fa24a20 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.592330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.596750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.596750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.596750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.596750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.597544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.707646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:53.725002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 928/1841] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:53 2022] POST /identity/v3/projects => generated 361 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:53.726901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.745341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1d015-93d0-49c5-8e1c-0c93c1d8b4c3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.809105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.809640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.809917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.810184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.811059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.813758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.848218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:53.909812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 914/1842] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:53 2022] POST /identity/v3/users => generated 281 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:53.914148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.925419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0979ca-f08e-4981-b255-a0cc8b9c633c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.929990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.930631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.930907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.931179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.931854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.933761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.946234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 929/1843] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:53.948773 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:53.966886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7f885-a103-4d82-bc30-68e023324e0e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:53.973066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:53.973713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:53.974054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:53.974264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:53.975053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:53.977390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e62f12-b6c1-4ecb-80f5-68692533f68b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:53.993181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 915/1844] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:53.997404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:54.008400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73313ecd-d243-4035-9337-67e8f28b5120 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:54.013775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:54.015051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:54.015051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:54.015051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] PATH_INFO: `/v3/projects/e44342dcec094b31aa682a45f00cbadd/users/3be60a7983d040e5b6f514c76806b6f4/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:54.030157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44342dcec094b31aa682a45f00cbadd, user_id=3be60a7983d040e5b6f514c76806b6f4, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:54.032952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:54.130201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 930/1845] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:53 2022] PUT /identity/v3/projects/e44342dcec094b31aa682a45f00cbadd/users/3be60a7983d040e5b6f514c76806b6f4/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:54.142699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:54.143110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:54.143546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:54.189889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] MFA Rules not processed for user `3be60a7983d040e5b6f514c76806b6f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:54.191719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:54.345246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 916/1846] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:54.642839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:54.644530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:54.701213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf781b-c556-4bdc-8b6a-f97c7500d065 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:54.704959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:54.705597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:54.705880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:54.706145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:54.715078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:54.716516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:54.727267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 931/1847] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:54.743118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:54.805177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08199e-8890-4b3e-97bb-580b201a2310 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:54.813848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:54.813848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:54.813848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:54.813848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] PATH_INFO: `/v3/users/3be60a7983d040e5b6f514c76806b6f4` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:54.814289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be60a7983d040e5b6f514c76806b6f4)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:54.814722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:54.872841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:54.874460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:54.888916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be5552-3971-44aa-a5b4-54be70a06fd6 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:54.894017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:54.894736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:54.895044 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:54.895323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] PATH_INFO: `/v3/users/cecba3cdd6324cd6944e82dc82ee6e18` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:54.896500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cecba3cdd6324cd6944e82dc82ee6e18)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:54.898473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:55.013031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be60a7983d040e5b6f514c76806b6f4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.019672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be60a7983d040e5b6f514c76806b6f4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.038945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cecba3cdd6324cd6944e82dc82ee6e18'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.050811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cecba3cdd6324cd6944e82dc82ee6e18'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.057483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cecba3cdd6324cd6944e82dc82ee6e18'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.058164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cecba3cdd6324cd6944e82dc82ee6e18'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.058867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a88ed91d-1cf7-49b3-8fbc-1bd3418f9108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cecba3cdd6324cd6944e82dc82ee6e18'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.093978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be60a7983d040e5b6f514c76806b6f4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.095261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be60a7983d040e5b6f514c76806b6f4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.107508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9bd57d27-a200-4d7f-8e1e-f87cba9e4c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be60a7983d040e5b6f514c76806b6f4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:55.114047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 917/1848] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:54 2022] DELETE /identity/v3/users/3be60a7983d040e5b6f514c76806b6f4 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:55.118610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.118895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.119171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.138143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:55.139425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.152570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 932/1849] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:54 2022] DELETE /identity/v3/users/cecba3cdd6324cd6944e82dc82ee6e18 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:55.158561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.158981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.159653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.181163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:55.182356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.202639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 918/1850] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:55.212705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:55.214030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.249979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 933/1851] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:55.257835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:55.259400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.271391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b2bdb-c304-46fc-9f49-7d3d6db113d0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:55.314878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2cfc6-f443-4c0b-a3c3-fb7188fa2475 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:55.318501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:55.319182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.319465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.319625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:55.319745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.320236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.320511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.320782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.329252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:55.330707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:55.330837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:55.332446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:55.340627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 934/1852] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:55.342335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 919/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:29:55.791872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:55.899739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:55.901242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.910367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:55.916790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d26a9-8eae-40b9-bd74-bf1ae15221ff service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:55.917722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:55.922115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:55.923051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.923562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.923981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] PATH_INFO: `/v3/projects/e44342dcec094b31aa682a45f00cbadd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.925356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44342dcec094b31aa682a45f00cbadd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:55.938326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ecba0-c210-41fa-b42d-e2144b54da96 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:55.939240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:55.943521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:55.944161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:55.944441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:55.944722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] PATH_INFO: `/v3/projects/6010c6661ac54fc185be5dd21827876b` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:55.946056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6010c6661ac54fc185be5dd21827876b)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:55.948079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:55.959615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:56.087111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6010c6661ac54fc185be5dd21827876b', 'request_id': 'req-41d49a2f-c62e-4071-b8ef-b82c8582a75e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.092837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6010c6661ac54fc185be5dd21827876b', 'request_id': 'req-41d49a2f-c62e-4071-b8ef-b82c8582a75e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.100747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44342dcec094b31aa682a45f00cbadd', 'request_id': 'req-570d7dc4-6c86-4577-adcd-998a0ccdf224'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.208843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6010c6661ac54fc185be5dd21827876b', 'request_id': 'req-41d49a2f-c62e-4071-b8ef-b82c8582a75e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.220791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] The token cache is being invalidate because project 6010c6661ac54fc185be5dd21827876b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:56.221142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.221916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.223430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 920/1854] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:55 2022] DELETE /identity/v3/projects/6010c6661ac54fc185be5dd21827876b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:56.228371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.229699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:56.283237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44342dcec094b31aa682a45f00cbadd', 'request_id': 'req-570d7dc4-6c86-4577-adcd-998a0ccdf224'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.289384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44342dcec094b31aa682a45f00cbadd', 'request_id': 'req-570d7dc4-6c86-4577-adcd-998a0ccdf224'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.297685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d49a2f-c62e-4071-b8ef-b82c8582a75e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.301269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] The token cache is being invalidate because project e44342dcec094b31aa682a45f00cbadd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:56.301562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.302235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-570d7dc4-6c86-4577-adcd-998a0ccdf224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.303747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 935/1855] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:55 2022] DELETE /identity/v3/projects/e44342dcec094b31aa682a45f00cbadd => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:56.308429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba6e4f-8945-4c43-b79b-ccfe2433ce24 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.308711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba6e4f-8945-4c43-b79b-ccfe2433ce24 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.308986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba6e4f-8945-4c43-b79b-ccfe2433ce24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.328711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7dba6e4f-8945-4c43-b79b-ccfe2433ce24 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:56.329911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7dba6e4f-8945-4c43-b79b-ccfe2433ce24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:56.350676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.351321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.354697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.354697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] PATH_INFO: `/v3/users/9ea725380bf44fbaadbe7095db441613` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.354697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea725380bf44fbaadbe7095db441613)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.355387 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.391795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 936/1856] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:56.399399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.399685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.399961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.419276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:56.420717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:56.429235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 937/1857] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:56.433244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.442884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045608c-0267-402c-a9da-e7621f7f3cb5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.447567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.448141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.448424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.448699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.449340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:56.449645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.451279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.463698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 938/1858] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:29:56.467890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.478266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e579-2b93-4627-b203-ed1e2ab334e0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.483233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.483850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.484124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.484391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.485072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.487056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.571356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea725380bf44fbaadbe7095db441613'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.578369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea725380bf44fbaadbe7095db441613'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.596666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 939/1859] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:56 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:56.597681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.635180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea725380bf44fbaadbe7095db441613'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.635873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea725380bf44fbaadbe7095db441613'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.652674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea725380bf44fbaadbe7095db441613'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:56.663357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 921/1860] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:29:56 2022] DELETE /identity/v3/users/9ea725380bf44fbaadbe7095db441613 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:56.664391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6082ec-c400-43f9-97ee-c24ccf07f332 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.761626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.762306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.762709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.763095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.763719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.765813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.888939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 940/1861] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:56 2022] POST /identity/v3/users => generated 288 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:56.889045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.890504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:56.908324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930c703-d71d-4ba8-876b-fa7dfdc62665 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.922977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.923598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.923876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.924145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.924967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.926797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.941027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 922/1862] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:56.942553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.952872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba11d1-b32a-4b74-b84a-dfdc3b4d79d5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:56.959880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:56.960868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:56.960868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:56.963437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:56.963437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:56.963712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf728e5-81d6-4748-bb34-35695141ffba admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:56.978520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 941/1863] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:29:56.981144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:56.992401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc6b9c-79bc-49eb-82ea-b3fa1b9bc178 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.001158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.001837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.002114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.002386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] PATH_INFO: `/v3/projects/eb630b09db1f471494303f0a2916a1b1/users/0074f7901cc34f3bbec1e88be8a8c9ec/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.020102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb630b09db1f471494303f0a2916a1b1, user_id=0074f7901cc34f3bbec1e88be8a8c9ec, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.023306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.122617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 923/1864] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:56 2022] PUT /identity/v3/projects/eb630b09db1f471494303f0a2916a1b1/users/0074f7901cc34f3bbec1e88be8a8c9ec/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:57.134833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.135116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.135386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.164040 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] MFA Rules not processed for user `0074f7901cc34f3bbec1e88be8a8c9ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:57.165268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:57.227881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 942/1865] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:57.232596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.288898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.293651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:57.325776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c377eb7-0cce-492c-824d-19c97bab9117 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.334541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.335292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.335640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.335982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.337023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.339182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.386228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f72a3-e1c8-4548-8697-d4217ef32d1a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.391551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.392193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.392477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.392752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] PATH_INFO: `/v3/projects/23d9adcd82494cc894bd900ac97e07e2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.393895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d9adcd82494cc894bd900ac97e07e2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.395861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.433426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 924/1866] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:57.437487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.505548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d9adcd82494cc894bd900ac97e07e2', 'request_id': 'req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:57.512452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26152831-b897-4db1-be52-6c5fd1e5b8ea admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.571619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.572331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.572680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.573162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.573908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.575929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.602198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d9adcd82494cc894bd900ac97e07e2', 'request_id': 'req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:57.608621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d9adcd82494cc894bd900ac97e07e2', 'request_id': 'req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:57.621020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] The token cache is being invalidate because project 23d9adcd82494cc894bd900ac97e07e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:29:57.621372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:57.622065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1bb96ea6-0365-4cd0-8374-5f80d13c2ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:29:57.623614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 943/1867] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:29:57 2022] DELETE /identity/v3/projects/23d9adcd82494cc894bd900ac97e07e2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:57.628805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00154b-cf76-4607-a0b1-4b92f60eea89 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.629172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00154b-cf76-4607-a0b1-4b92f60eea89 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.629520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00154b-cf76-4607-a0b1-4b92f60eea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.652795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1e00154b-cf76-4607-a0b1-4b92f60eea89 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:57.654020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1e00154b-cf76-4607-a0b1-4b92f60eea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:57.662833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 944/1868] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:57.670221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.675037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.675037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.689497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:57.690700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:57.699976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 945/1869] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:29:57.702861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 925/1870] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:57.706834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.707281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.708735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:57.721657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e5e5f-2cf3-47f5-9184-8eecdb234fca None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.722883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7eed5-c609-46e4-86ba-c791e3e988ce admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.727013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.727788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.728174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.728448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.729252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.729866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.730573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.730854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.733790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.733919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.733919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:29:57.733919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.733919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.754822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 946/1871] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:57.772220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 926/1872] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:29:57.773592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.777415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.790252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d1e5c-7444-40c4-ba25-e99ed6591965 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.791310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b7c6-6dd9-481a-b134-51039c044cfb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.797110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.802783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.803354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.803354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.803529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.803615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.803708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.803708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.803858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.803938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.804025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.804113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.815822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 947/1873] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:57.842893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:57.874431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70ffdd-81f5-4ec2-ba75-395eeea6f974 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:57.884496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:57.885182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:57.885603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:57.885883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] PATH_INFO: `/v3/projects/41dd4662f9bd4d9aafa230e92dafa85e/users/0192aca24edb4293a0bbe47acdd906b1/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:57.904900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dd4662f9bd4d9aafa230e92dafa85e, user_id=0192aca24edb4293a0bbe47acdd906b1, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:57.908487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc257e7c-4ff2-4f35-8465-ad0512d60d02 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:57.986114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 927/1874] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:57.987282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.019964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 948/1875] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:57 2022] PUT /identity/v3/projects/41dd4662f9bd4d9aafa230e92dafa85e/users/0192aca24edb4293a0bbe47acdd906b1/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:58.031989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.032431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.032713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.141050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94731b64-1f1a-4eed-9563-e188c2aa1959 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.145922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] MFA Rules not processed for user `0192aca24edb4293a0bbe47acdd906b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:58.149978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:58.222657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.223984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.224426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.224857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.225766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.228194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.276817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 949/1876] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:58.283783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.298147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8d5c4-a3bc-462c-b75c-19d0ef80758a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.303576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.307755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.307755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.307755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.307755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.308515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.387026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 928/1877] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:57 2022] POST /identity/v3/users => generated 290 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:29:58.393216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.416707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807f971-0512-4073-b4dc-c14a85cb9e42 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.424747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.425689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.426319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.426845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.428944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.431628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.445802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 929/1878] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:58.449955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.461889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bab1b-9827-4022-bc0e-407aad8fd8fa admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.468130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.478008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.478008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.478008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.478008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.478008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7837c6-d35e-4717-bd6e-a3fc09b89c1e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.481415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 950/1879] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:58 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:58.482654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.487764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 930/1880] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:58.490007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374335-4f85-42bd-bd1e-616be53aeec1 None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.490814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374335-4f85-42bd-bd1e-616be53aeec1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.491293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db374335-4f85-42bd-bd1e-616be53aeec1 None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.493151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 931/1881] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:29:58.494147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db374335-4f85-42bd-bd1e-616be53aeec1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.684020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:58.685411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db374335-4f85-42bd-bd1e-616be53aeec1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.698393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.699119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.699404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.699676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] PATH_INFO: `/v3/projects/0e2e4021631242f59f6e211af7bed66b/users/b11c1924660b4cd59beec8368467bede/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.700148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661f79e-0d5d-4e94-8520-69691ae2dc48 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:58.714361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:58.715030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.716070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.716344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.717059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.719752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9e9cc-ae24-4cfd-9465-5a53cd342f96 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.750893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2e4021631242f59f6e211af7bed66b, user_id=b11c1924660b4cd59beec8368467bede, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:58.756417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:58.767866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:29:58.880316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 951/1882] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:58 2022] POST /identity/v3/users => generated 294 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:58.881036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:58.881326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:58.881601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:58.897108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 932/1883] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:58 2022] PUT /identity/v3/projects/0e2e4021631242f59f6e211af7bed66b/users/b11c1924660b4cd59beec8368467bede/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:58.898896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:58.911168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:58.912686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.011485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450fb7d-5b5d-47f1-b854-a425036e2136 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.035940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.037104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.039139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.039582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.040416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.044393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec446028-23d7-4471-b69c-a8c1af3712b0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.066763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 933/1884] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:59.068150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.068436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.068706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.071291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 952/1885] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:29:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:59.075704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.095564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f099-2274-457a-bb83-a16123cc5c65 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.101878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.102598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.102880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.103155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] PATH_INFO: `/v3/projects/187485d1f0cb4842ba9f9188c216c883/users/dba62e74621a4bb0ade54b0f8aa5d2ec/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.105222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] MFA Rules not processed for user `b11c1924660b4cd59beec8368467bede`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:59.107603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.121269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187485d1f0cb4842ba9f9188c216c883, user_id=dba62e74621a4bb0ade54b0f8aa5d2ec, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.124207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.189533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 934/1886] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:59.190775 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.192127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.212628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4c09cc-335a-4d94-8b46-ab9f75b6ec89 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.217229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 953/1887] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:59 2022] PUT /identity/v3/projects/187485d1f0cb4842ba9f9188c216c883/users/dba62e74621a4bb0ade54b0f8aa5d2ec/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:29:59.220891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.305630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.306584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.307257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.307747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.327297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bd5ba-166c-4251-8edd-ad1b44d59f07 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.334715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.335706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.335706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.335971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.338893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.340387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.344633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.347209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.374002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 935/1888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:29:59.376025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.390686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f164b9-c5f6-4d7f-ab17-678a6d045029 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.397475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.399529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.399938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.400274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.401030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.403117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.436633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 936/1889] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:59.445617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.449002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 954/1890] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:29:59 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:59.451257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.465860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e427614-36f2-4fcf-b92d-474f0d8d7f28 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.555147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.555916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.556837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.557117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] PATH_INFO: `/v3/projects/187485d1f0cb4842ba9f9188c216c883/users/dba62e74621a4bb0ade54b0f8aa5d2ec/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.559404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187485d1f0cb4842ba9f9188c216c883, user_id=dba62e74621a4bb0ade54b0f8aa5d2ec, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.563662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bbefa4-fa83-4782-acef-5d4f3a0ad9a1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.591106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 937/1891] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:59 2022] PUT /identity/v3/projects/187485d1f0cb4842ba9f9188c216c883/users/dba62e74621a4bb0ade54b0f8aa5d2ec/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:29:59.593453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9f25-d065-4573-8def-804073d46821 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.593453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9f25-d065-4573-8def-804073d46821 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.593453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9f25-d065-4573-8def-804073d46821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.604596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27d8bd-bb67-4d86-a38c-a19332cfb443 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.633364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-abfb9f25-d065-4573-8def-804073d46821 None None] MFA Rules not processed for user `dba62e74621a4bb0ade54b0f8aa5d2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:29:59.637522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-abfb9f25-d065-4573-8def-804073d46821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.760324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.762505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.762970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.763406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.764283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.767764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.797691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 938/1892] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:29:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:29:59.811160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9f25-d065-4573-8def-804073d46821 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.817311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-abfb9f25-d065-4573-8def-804073d46821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.894800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 955/1893] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:29:59 2022] POST /identity/v3/users => generated 296 bytes in 444 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:29:59.897472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.926675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c93648-1b7d-4d76-9574-50b5af9cace3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.942914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.943602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.943889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.944159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.944965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:29:59.946610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:29:59.950673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9f25-d065-4573-8def-804073d46821 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:29:59.954469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:29:59.955144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:29:59.955425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:29:59.955791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:29:59.958157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:29:59.982820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 956/1894] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:29:59.983895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:29:59.998372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8799a2-a6b1-4c97-aa6c-437ea679009c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:00.014475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:00.015168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:00.016524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:00.016814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] PATH_INFO: `/v3/projects/b98251d312114baebef93143bafba19f/users/1b91fe61fc9a48479febf125d3d51cbf/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:00.041590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:00.043476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:00.053916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98251d312114baebef93143bafba19f, user_id=1b91fe61fc9a48479febf125d3d51cbf, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:00.056589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 939/1895] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:29:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:00.060011 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:00.156491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 957/1896] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:29:59 2022] PUT /identity/v3/projects/b98251d312114baebef93143bafba19f/users/1b91fe61fc9a48479febf125d3d51cbf/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:00.160716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:00.175204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb167e-0dfa-4d4b-b678-0a38ef71a6d6 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:00.334323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:00.335311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:00.335702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:00.351649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:00.353257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:00.355721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb34e72-dea5-48b8-88f1-f398808d7f54 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:00.377658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 940/1897] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:00.381581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:00.394144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487958c-7b0e-4d5b-8414-b30c08908231 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:00.405971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:00.407194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:00.407473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:00.407748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] PATH_INFO: `/v3/projects/b98251d312114baebef93143bafba19f/users/1b91fe61fc9a48479febf125d3d51cbf/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:00.410957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98251d312114baebef93143bafba19f, user_id=1b91fe61fc9a48479febf125d3d51cbf, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:00.413813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:00.467620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 958/1898] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:00 2022] PUT /identity/v3/projects/b98251d312114baebef93143bafba19f/users/1b91fe61fc9a48479febf125d3d51cbf/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:00.482651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:00.483012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:00.483244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:00.571612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] MFA Rules not processed for user `1b91fe61fc9a48479febf125d3d51cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:00.573214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:00.759185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 941/1899] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:00.882813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:01.124943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:01.183589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614df1c7-8ed2-403d-a954-50f36013ebf2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:01.187204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:01.187812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:01.188090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:01.188360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:01.197383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:01.198939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:01.221282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 959/1900] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:01.934886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:02.649848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:02.670368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5b12c-e50e-45a9-a230-646783f96990 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:02.690645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b91fe61fc9a48479febf125d3d51cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98251d312114baebef93143bafba19f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:02.694721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:02.695330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:02.695837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:02.696739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:02.697252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:02.703383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:02.741136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:02.742685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:02.748472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 942/1901] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:02.754907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:02.770028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070b6d4-fe2a-4721-8316-c6f1c1a14841 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:02.775141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b91fe61fc9a48479febf125d3d51cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98251d312114baebef93143bafba19f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:02.775874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:02.776224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:02.776574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:02.777442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:02.779655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:02.860150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78435-6991-4271-931d-28f0b50e369b service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:02.864884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:02.865526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:02.865805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:02.866078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] PATH_INFO: `/v3/users/e48cd79fbd5047ea930d5aea116853bc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:02.867261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48cd79fbd5047ea930d5aea116853bc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:02.869244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:02.887910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 943/1902] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:02 2022] POST /identity/v3/projects => generated 354 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:02.887910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:02.889003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:02.947958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c80-1869-4b66-99a0-f4b682562266 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:02.951487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:02.952090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:02.952367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:02.952660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:03.007684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:03.009157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:03.018747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 944/1903] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:03.019553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:03.084548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c6558-2336-422c-b81e-3892690ba9f4 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:03.084758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48cd79fbd5047ea930d5aea116853bc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.089320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b91fe61fc9a48479febf125d3d51cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98251d312114baebef93143bafba19f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:03.089936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:03.090220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:03.090526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:03.091178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:03.091484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:03.093100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9278f-c457-46c1-bacd-1d29a4b47a37 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:03.096748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48cd79fbd5047ea930d5aea116853bc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.103620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48cd79fbd5047ea930d5aea116853bc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.104338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48cd79fbd5047ea930d5aea116853bc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.104955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48cd79fbd5047ea930d5aea116853bc'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.105240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 945/1904] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:03.207175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 960/1905] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:02 2022] DELETE /identity/v3/users/e48cd79fbd5047ea930d5aea116853bc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:03.211641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:03.211932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:03.212205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:03.233012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:03.235116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:03.302381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 946/1906] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:03.314668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:03.316320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:03.393310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec922-3298-47b1-aa1f-1a82223aac12 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:03.399254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:03.399966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:03.400154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:03.400432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:03.411248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:03.412723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:03.432183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 961/1907] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:03.486761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:03.488165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:03.602234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebeb92b-867c-40f0-9964-480e4c79e449 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:03.610160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b91fe61fc9a48479febf125d3d51cbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98251d312114baebef93143bafba19f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:03.610955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:03.611299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:03.615638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] PATH_INFO: `/v3/projects/8b4ef97cc08e4543987457354b32f733` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:03.615638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8b4ef97cc08e4543987457354b32f733)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:03.615638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:03.761127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4ef97cc08e4543987457354b32f733', 'request_id': 'req-6811f863-7842-4825-9254-b59823007158'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.767194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4ef97cc08e4543987457354b32f733', 'request_id': 'req-6811f863-7842-4825-9254-b59823007158'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.865251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4ef97cc08e4543987457354b32f733', 'request_id': 'req-6811f863-7842-4825-9254-b59823007158'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.878155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] The token cache is being invalidate because project 8b4ef97cc08e4543987457354b32f733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:03.878481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.879172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:03.880694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 947/1908] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:03 2022] DELETE /identity/v3/projects/8b4ef97cc08e4543987457354b32f733 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:03.888397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:03.958839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:03.964763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:03.999260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b47fdc-9159-4703-96c9-c0ab862248f9 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:04.065446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:04.106132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:04.107266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:04.107648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:04.108432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] PATH_INFO: `/v3/projects/9a996409ea3e4d39828895a711a40a9e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:04.109904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a996409ea3e4d39828895a711a40a9e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:04.112317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:04.344008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a996409ea3e4d39828895a711a40a9e', 'request_id': 'req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.428623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a996409ea3e4d39828895a711a40a9e', 'request_id': 'req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.435873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a996409ea3e4d39828895a711a40a9e', 'request_id': 'req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.449147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] The token cache is being invalidate because project 9a996409ea3e4d39828895a711a40a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:04.449437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.450752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.454559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 962/1909] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:03 2022] DELETE /identity/v3/projects/9a996409ea3e4d39828895a711a40a9e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:04.456844 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:04.458266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:04.471938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811f863-7842-4825-9254-b59823007158 tempest-VolumeQuotaClassesTest-463658671 tempest-VolumeQuotaClassesTest-463658671-project-admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:04.546335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:04.547560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:04.547839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:04.548116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] PATH_INFO: `/v3/users/f256f881dd9045b0a9cb3638a7e84258` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:04.549351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f256f881dd9045b0a9cb3638a7e84258)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:04.551414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:04.810975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f256f881dd9045b0a9cb3638a7e84258'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.820457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f256f881dd9045b0a9cb3638a7e84258'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.911903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f256f881dd9045b0a9cb3638a7e84258'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.912787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f256f881dd9045b0a9cb3638a7e84258'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.931109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f256f881dd9045b0a9cb3638a7e84258'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:04.940848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 948/1910] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:04 2022] DELETE /identity/v3/users/f256f881dd9045b0a9cb3638a7e84258 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:05.083497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:05.085625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.099341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c25ce-c90a-4784-9239-f061b4dc3cdb admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:05.161499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:05.162135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.162426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.162716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] PATH_INFO: `/v3/users/b11c1924660b4cd59beec8368467bede` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.163987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11c1924660b4cd59beec8368467bede)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:05.165944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:05.321134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:05.322618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.396793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cce54-0fe5-4d06-a571-457508717344 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:05.400339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:05.400938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.401212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.401482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.403213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.463246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:05.464741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:05.474580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 949/1911] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:05.551103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:05.552552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.554651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11c1924660b4cd59beec8368467bede'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.570654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11c1924660b4cd59beec8368467bede'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.582135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11c1924660b4cd59beec8368467bede'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.583329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11c1924660b4cd59beec8368467bede'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.584249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d6231f01-de73-4959-bf12-811c5555f176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11c1924660b4cd59beec8368467bede'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.585953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f1c87-2012-44ed-bbf6-5a3c551abfb3 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:05.610303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:05.610998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.611279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.611557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] PATH_INFO: `/v3/projects/d2272721bf874bcd839a865884f523c8` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.622830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2272721bf874bcd839a865884f523c8)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:05.625203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:05.669631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 963/1912] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:05 2022] DELETE /identity/v3/users/b11c1924660b4cd59beec8368467bede => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:05.675345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.676244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.676244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.697450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:05.698717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.772595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 964/1913] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:05.785381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:05.787807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:05.808629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2272721bf874bcd839a865884f523c8', 'request_id': 'req-6e6c2914-f44e-42b5-a371-836c53deb6aa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.815272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2272721bf874bcd839a865884f523c8', 'request_id': 'req-6e6c2914-f44e-42b5-a371-836c53deb6aa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.854999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6df64-a9de-4912-b18e-5ed7ea4b3ba5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:05.904188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:05.904829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.904935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2272721bf874bcd839a865884f523c8', 'request_id': 'req-6e6c2914-f44e-42b5-a371-836c53deb6aa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.905110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.905378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.935845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] The token cache is being invalidate because project d2272721bf874bcd839a865884f523c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:05.936149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.936827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e6c2914-f44e-42b5-a371-836c53deb6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:05.941278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 950/1914] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:05 2022] DELETE /identity/v3/projects/d2272721bf874bcd839a865884f523c8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:05.946059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9309-87d8-4ea2-a350-372816f82575 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:05.946537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9309-87d8-4ea2-a350-372816f82575 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:05.947084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f9309-87d8-4ea2-a350-372816f82575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:05.988463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-574f9309-87d8-4ea2-a350-372816f82575 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:05.990007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-574f9309-87d8-4ea2-a350-372816f82575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:06.000228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.001669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 951/1915] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:06.002392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.011316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.011896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.012449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.014867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 965/1916] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:06.041654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:06.043276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:06.054562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 952/1917] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:06.058828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.060240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:06.073512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54620644-897c-4106-9878-264f98ed08a2 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.079819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:06.083369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.083993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.085285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.085285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.085727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:06.086031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.088302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.105348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 966/1918] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:06.106680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.119715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541c793-98ba-4729-bd12-a4def8b90ef1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.126013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.126687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.127172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.127452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.128118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.132063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.275681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 953/1919] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:06 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:06.277211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.288345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499dbdc-6fbb-4771-8cd8-d5b94f2376b3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.379713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.380746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.381554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.382012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.382905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.386295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.520876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 967/1920] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:06 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:06.523307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.550461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4261-735f-4b5a-b576-1a9d7dbfa918 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.566489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.566489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.566489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.566489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.566858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.566858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.585363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 954/1921] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:06.589878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.614548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea686bd-69de-4e90-9865-ad9104f26c61 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.616247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.616917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.617233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.617562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.618305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.620837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.644365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 968/1922] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:06.648830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.663984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c1a34-7006-4a30-8541-764ac232f684 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.669481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.671796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.671796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.671796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] PATH_INFO: `/v3/projects/9eb8e7ae881841c0afe23346de9e0761/users/c5527be4561b4ff3868c1b3b9a81cbcc/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.696846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb8e7ae881841c0afe23346de9e0761, user_id=c5527be4561b4ff3868c1b3b9a81cbcc, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.700473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f46b7-5640-484f-b739-8bd8c86c9fb5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.866683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 955/1923] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:06 2022] PUT /identity/v3/projects/9eb8e7ae881841c0afe23346de9e0761/users/c5527be4561b4ff3868c1b3b9a81cbcc/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:06.867725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:06.869426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:06.883608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.883896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.884170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.885832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65de7d9-9fa9-4808-bb08-86be0aa09358 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:06.913252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] MFA Rules not processed for user `c5527be4561b4ff3868c1b3b9a81cbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:06.914541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:06.935292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:06.945952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:06.946615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:06.946894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:06.947161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] PATH_INFO: `/v3/projects/0e2e4021631242f59f6e211af7bed66b` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:06.948418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2e4021631242f59f6e211af7bed66b)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:06.950313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:06.981579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 956/1924] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:06.986044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.007627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfe5b3-ac21-42c7-a66c-1c844cbcae79 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.020584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.024939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.024939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.024939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.024939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.024939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.058624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2e4021631242f59f6e211af7bed66b', 'request_id': 'req-7167f53d-028c-44a0-bcc3-02d9d325876b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.128149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2e4021631242f59f6e211af7bed66b', 'request_id': 'req-7167f53d-028c-44a0-bcc3-02d9d325876b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.137069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 957/1925] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:06 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:07.137690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2e4021631242f59f6e211af7bed66b', 'request_id': 'req-7167f53d-028c-44a0-bcc3-02d9d325876b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.141248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.160901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] The token cache is being invalidate because project 0e2e4021631242f59f6e211af7bed66b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:07.161201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.162611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.166473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 969/1926] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:06 2022] DELETE /identity/v3/projects/0e2e4021631242f59f6e211af7bed66b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:07.168583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.170556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:07.219783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff6e18-db51-4709-8213-4272bc7df765 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.225767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.226568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.226915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.227588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.228351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.231080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.261277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167f53d-028c-44a0-bcc3-02d9d325876b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.266429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.267551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.267843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.268122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] PATH_INFO: `/v3/users/1b91fe61fc9a48479febf125d3d51cbf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.269367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b91fe61fc9a48479febf125d3d51cbf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.272924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.457280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 958/1927] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:07 2022] POST /identity/v3/users => generated 294 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:07.458522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.460168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:07.472535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7e694-439d-4b34-a0a6-397c0c6a1ca3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.477356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.477954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.478237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.478547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.479192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.480893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.491343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 959/1928] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:07.495323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.506039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df5bb3-680c-4bcb-b190-85b3e4ac98fc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.510667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.511391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.511714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.511985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.512875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.514338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.525679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 960/1929] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:07.530436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.542131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ee697-d8bc-445e-866e-63114726a5cc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:07.551083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:07.551083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.551083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.551083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] PATH_INFO: `/v3/projects/92ebdbeaf4b2414d9c0f9207e182bcdb/users/ca498d5a6fb74a89b71ea7756a6aa4ae/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.564325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ebdbeaf4b2414d9c0f9207e182bcdb, user_id=ca498d5a6fb74a89b71ea7756a6aa4ae, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:07.567036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff343a14-bccf-4ba3-b847-0e36e30bb974 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:07.610733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b91fe61fc9a48479febf125d3d51cbf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.624256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b91fe61fc9a48479febf125d3d51cbf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.648679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b91fe61fc9a48479febf125d3d51cbf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.649360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b91fe61fc9a48479febf125d3d51cbf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.649979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b91fe61fc9a48479febf125d3d51cbf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:07.665726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 961/1930] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:07 2022] PUT /identity/v3/projects/92ebdbeaf4b2414d9c0f9207e182bcdb/users/ca498d5a6fb74a89b71ea7756a6aa4ae/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:07.678114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:07.678570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:07.678912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:07.723227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] MFA Rules not processed for user `ca498d5a6fb74a89b71ea7756a6aa4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:07.725546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:07.755786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 970/1931] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:07 2022] DELETE /identity/v3/users/1b91fe61fc9a48479febf125d3d51cbf => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:07.872546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 962/1932] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:07.878660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:07.881155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.014627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6c15e-4b42-406f-860b-4429a9491a7c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.017299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.017921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.018191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.018489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.019119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.021027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.169903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 971/1933] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:07 2022] POST /identity/v3/projects => generated 387 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:08.171194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.235439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac521b0b-3e74-4855-8bd9-d68b31023aa9 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.290710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.291318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.291660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.291932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.292562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.294532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.345183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.347793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.361828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e724-a28b-4481-9a61-215fdcd83a16 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.366801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.367428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.367709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.367981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] PATH_INFO: `/v3/projects/b98251d312114baebef93143bafba19f` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.370462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98251d312114baebef93143bafba19f)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.372485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.397163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 963/1934] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:08 2022] POST /identity/v3/users => generated 300 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:08.401413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.421363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab1f3a-9667-47c6-84b4-c8696eb0038f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.426884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.427612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.427988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.428359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.429125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.431635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.445534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 964/1935] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:08.450099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.461127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8687b-ce0f-4408-bdca-71d2b5767561 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.465956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.466652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.466933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.467207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] PATH_INFO: `/v3/projects/36d150afa2084c7b9b604e533ec486e8/users/ee54019c09cb44f2b986611234abdf9c/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.487310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d150afa2084c7b9b604e533ec486e8, user_id=ee54019c09cb44f2b986611234abdf9c, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.489192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.614914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 965/1936] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:08 2022] PUT /identity/v3/projects/36d150afa2084c7b9b604e533ec486e8/users/ee54019c09cb44f2b986611234abdf9c/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:08.615008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98251d312114baebef93143bafba19f', 'request_id': 'req-114f3c21-0464-47be-998d-b9d4551d6e23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:08.617618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.680142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399860b-7b0e-4759-8e4d-89202519d657 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.684974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.685586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.685864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.686151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.686820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.688964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.701261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 966/1937] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:08.706320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.716310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258df1-cd35-497e-a682-9fb0fc1ab139 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.720933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.721576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.721854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.722120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] PATH_INFO: `/v3/projects/36d150afa2084c7b9b604e533ec486e8/users/ee54019c09cb44f2b986611234abdf9c/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.724295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d150afa2084c7b9b604e533ec486e8, user_id=ee54019c09cb44f2b986611234abdf9c, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.726792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272710a-62a3-48ee-b162-2786477c438f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:08.738798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 967/1938] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:08 2022] PUT /identity/v3/projects/36d150afa2084c7b9b604e533ec486e8/users/ee54019c09cb44f2b986611234abdf9c/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:08.750316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d81d1-b4ec-4f77-9325-9df8b367cd92 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.750629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d81d1-b4ec-4f77-9325-9df8b367cd92 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.750898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d81d1-b4ec-4f77-9325-9df8b367cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.762168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98251d312114baebef93143bafba19f', 'request_id': 'req-114f3c21-0464-47be-998d-b9d4551d6e23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:08.771572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98251d312114baebef93143bafba19f', 'request_id': 'req-114f3c21-0464-47be-998d-b9d4551d6e23'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:08.782326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e86d81d1-b4ec-4f77-9325-9df8b367cd92 None None] MFA Rules not processed for user `ee54019c09cb44f2b986611234abdf9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:08.783601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e86d81d1-b4ec-4f77-9325-9df8b367cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.790723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] The token cache is being invalidate because project b98251d312114baebef93143bafba19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:08.791144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:08.792055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-114f3c21-0464-47be-998d-b9d4551d6e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:08.795131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 972/1939] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:08 2022] DELETE /identity/v3/projects/b98251d312114baebef93143bafba19f => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:08.800466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.800815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.801096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.837687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:08.838949 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.907302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 968/1940] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:08.912955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 973/1941] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:08.920539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.920822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.921090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.941416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:08.942661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.951363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 969/1942] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:08.956274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.966117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615fc4c-c50e-41db-86d2-34934536038a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:08.976246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:08.980646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:08.980646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:08.980646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:08.980646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:08.980646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:08.981072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:08.981072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:08.981236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.001160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 974/1943] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:09.002239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.012219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509be6b-6b11-41ff-a23c-ecb9067232f8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.018136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.018633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.018911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.020324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.020324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.024000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.050395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c525c-91c2-4e4a-acda-c6a6e5db30ac None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.055271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.056270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.056270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.060468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.066269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.067767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.078228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 970/1944] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:09.082554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:09.129541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 975/1945] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:09 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:09.133502 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.146079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee6053-e643-4749-afd1-278f5206a638 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.173783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:09.241372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.242256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.245707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.246228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.247124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.250452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.379917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 971/1946] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:09 2022] POST /identity/v3/users => generated 295 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:09.384070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.395337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b9e5d-40a5-46b3-978a-6ad0a3d03b3a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.400247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.404309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.404309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.404309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.404309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.404309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.414074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 976/1947] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:09.418163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.428692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdef5e-7981-4d3b-92c2-753e014688cc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.433396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.434040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.434453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.434737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.435383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.437498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4e7cb-af2b-467a-81cb-80fab091ed4c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.453447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 972/1948] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:09.456106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.469321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e6994-0d10-4846-aaf6-3068d646e276 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.485412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.485412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.485412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.485412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] PATH_INFO: `/v3/projects/a27e760f29cc468183dbc4f44071d9b2/users/72f6fea027584de18e99802d564d4221/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.502153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a27e760f29cc468183dbc4f44071d9b2, user_id=72f6fea027584de18e99802d564d4221, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.504957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:09.668607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 977/1949] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:09 2022] PUT /identity/v3/projects/a27e760f29cc468183dbc4f44071d9b2/users/72f6fea027584de18e99802d564d4221/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:09.680601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.680891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.681170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.712191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] MFA Rules not processed for user `72f6fea027584de18e99802d564d4221`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:09.713423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:09.805981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 973/1950] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:09.806089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:09.883284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11c763-16fa-45f1-82c3-7440da91997c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:09.896275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:09.897017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:09.897437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:09.897806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:09.898622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:09.900659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.030108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 978/1951] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:09 2022] POST /identity/v3/projects => generated 389 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:10.034831 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.046053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21490fe-fd73-449c-92ec-2c77e04d0c86 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.116265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.117057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.117344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.117620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.118370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.120488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.200568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 974/1952] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:10 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:10.204701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.215408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee1d29-874a-447c-9c9d-27f55ae365e3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.220524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.221124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.222341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.222771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.223436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.225339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.238144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 979/1953] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:10.241193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.252401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5712fe8-77f0-4143-9757-5328c531669c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.258046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.258753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.262001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.262001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] PATH_INFO: `/v3/projects/61ead368719a49e7b56fb4b3d004e46e/users/4f2f09c3a8044bb18bc6b34c8641efab/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.283898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ead368719a49e7b56fb4b3d004e46e, user_id=4f2f09c3a8044bb18bc6b34c8641efab, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.288735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.399854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 975/1954] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:10 2022] PUT /identity/v3/projects/61ead368719a49e7b56fb4b3d004e46e/users/4f2f09c3a8044bb18bc6b34c8641efab/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:10.404143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.424558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6dd3b-92a3-4722-bd77-733ad8aaecf0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.550989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.551624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.551904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.552304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.552823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.557588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232988f-5e57-44cf-a9db-8c753c46970d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.598887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 980/1955] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:10.603297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.616585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9f109-e89f-46ec-a76f-c4c650ca1407 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.639128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.646166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.646166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.646166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] PATH_INFO: `/v3/projects/61ead368719a49e7b56fb4b3d004e46e/users/4f2f09c3a8044bb18bc6b34c8641efab/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.652081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ead368719a49e7b56fb4b3d004e46e, user_id=4f2f09c3a8044bb18bc6b34c8641efab, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.657244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.687641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 976/1956] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:10 2022] PUT /identity/v3/projects/61ead368719a49e7b56fb4b3d004e46e/users/4f2f09c3a8044bb18bc6b34c8641efab/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:10.700521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.701308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.701732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.754498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.766569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] MFA Rules not processed for user `4f2f09c3a8044bb18bc6b34c8641efab`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:10.768822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:10.851429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c066f-a3e2-436a-9491-cba278ac5ab8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.862973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.863858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.864239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.864527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 981/1957] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:10.864643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.868754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:10.895605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:10.897239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:10.931636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea9837-e4e6-42c1-8779-3fe0c10887e3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:10.942169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:10.942831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:10.943114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:10.943391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:10.953872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.955881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.963519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:10.965195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:10.967648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 977/1958] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:10.978470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 982/1959] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:11.077424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:11.128390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:12.116618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:12.118093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:12.183414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:12.220818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ccfca-00a8-49c4-bf71-5e90400ad08e service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:12.228070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:12.228662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.228944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.229221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] PATH_INFO: `/v3/users/0074f7901cc34f3bbec1e88be8a8c9ec` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.231803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0074f7901cc34f3bbec1e88be8a8c9ec)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:12.233941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:12.274670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae2587-6dc1-4675-ac92-55c5443c770b service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:12.280528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:12.281458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.281842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.282354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] PATH_INFO: `/v3/users/72f6fea027584de18e99802d564d4221` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.284462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f6fea027584de18e99802d564d4221)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:12.286846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:12.440202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0074f7901cc34f3bbec1e88be8a8c9ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.449799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0074f7901cc34f3bbec1e88be8a8c9ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.496466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f6fea027584de18e99802d564d4221'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.509364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f6fea027584de18e99802d564d4221'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.516423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f6fea027584de18e99802d564d4221'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.517107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f6fea027584de18e99802d564d4221'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.517744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-785f3763-3ed6-40ad-bd19-e63337b2accc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f6fea027584de18e99802d564d4221'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.527345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0074f7901cc34f3bbec1e88be8a8c9ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.527974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0074f7901cc34f3bbec1e88be8a8c9ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.540595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4bab4c2f-9ffd-46b9-94e8-fa14b8bd279e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0074f7901cc34f3bbec1e88be8a8c9ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:12.547113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 978/1960] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:12 2022] DELETE /identity/v3/users/0074f7901cc34f3bbec1e88be8a8c9ec => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:12.551424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.551701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.551978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.575457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:12.582452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:12.602076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 983/1961] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:12 2022] DELETE /identity/v3/users/72f6fea027584de18e99802d564d4221 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:12.607037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.607600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.608051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.630145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:12.631639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:12.660312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 979/1962] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:12.667262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:12.668674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:12.706503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 984/1963] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:12.713775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:12.715460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:12.732597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eb968-d436-4ed1-88ce-c845863b4a7f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:12.784990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25376f16-1b37-4b21-96c2-bcf2daa8e84b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:12.787390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:12.788027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.788290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.788637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.789342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:12.789951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:12.790226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:12.790577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:12.797922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:12.799423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:12.800342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:12.802026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:12.809444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 980/1964] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:12.812013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 985/1965] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:13.148847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:13.219892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:13.369217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:13.372453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:13.385376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf5604-668c-4624-b275-6a7c05482578 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:13.394204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:13.395415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:13.396053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:13.397453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] PATH_INFO: `/v3/projects/eb630b09db1f471494303f0a2916a1b1` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:13.398995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:13.400221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb630b09db1f471494303f0a2916a1b1)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:13.400605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:13.403262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:13.418899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfe190-1060-41fe-a7f8-2e461e9cfbf2 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:13.431272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:13.432382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:13.432668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:13.432950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] PATH_INFO: `/v3/projects/a27e760f29cc468183dbc4f44071d9b2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:13.434108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a27e760f29cc468183dbc4f44071d9b2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:13.436221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:13.531827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb630b09db1f471494303f0a2916a1b1', 'request_id': 'req-44958f21-1b43-49c9-9531-e240d2b024e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.539648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb630b09db1f471494303f0a2916a1b1', 'request_id': 'req-44958f21-1b43-49c9-9531-e240d2b024e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.561429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a27e760f29cc468183dbc4f44071d9b2', 'request_id': 'req-ee46a7b6-6718-42a7-93a2-d275153e0686'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.650931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb630b09db1f471494303f0a2916a1b1', 'request_id': 'req-44958f21-1b43-49c9-9531-e240d2b024e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.663456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] The token cache is being invalidate because project eb630b09db1f471494303f0a2916a1b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:13.663781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.664419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.665918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 981/1966] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:13 2022] DELETE /identity/v3/projects/eb630b09db1f471494303f0a2916a1b1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:13.669703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:13.671628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:13.734853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44958f21-1b43-49c9-9531-e240d2b024e2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:13.736629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a27e760f29cc468183dbc4f44071d9b2', 'request_id': 'req-ee46a7b6-6718-42a7-93a2-d275153e0686'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.742757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a27e760f29cc468183dbc4f44071d9b2', 'request_id': 'req-ee46a7b6-6718-42a7-93a2-d275153e0686'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.754573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] The token cache is being invalidate because project a27e760f29cc468183dbc4f44071d9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:13.754865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.755509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.759924 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 986/1967] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:13 2022] DELETE /identity/v3/projects/a27e760f29cc468183dbc4f44071d9b2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:13.760975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:13.762354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:13.792978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:13.793632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:13.795740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:13.795740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] PATH_INFO: `/v3/users/0192aca24edb4293a0bbe47acdd906b1` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:13.795740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0192aca24edb4293a0bbe47acdd906b1)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:13.798001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:13.833839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46a7b6-6718-42a7-93a2-d275153e0686 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:13.838524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:13.839161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:13.839442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:13.839720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] PATH_INFO: `/v3/users/4f2f09c3a8044bb18bc6b34c8641efab` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:13.840907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2f09c3a8044bb18bc6b34c8641efab)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:13.843553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:13.957430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0192aca24edb4293a0bbe47acdd906b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:13.964329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0192aca24edb4293a0bbe47acdd906b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.015828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2f09c3a8044bb18bc6b34c8641efab'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.024953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0192aca24edb4293a0bbe47acdd906b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.025588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0192aca24edb4293a0bbe47acdd906b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.029228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2f09c3a8044bb18bc6b34c8641efab'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.037347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2f09c3a8044bb18bc6b34c8641efab'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.038091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2f09c3a8044bb18bc6b34c8641efab'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.038756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2f09c3a8044bb18bc6b34c8641efab'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.038865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0192aca24edb4293a0bbe47acdd906b1'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.044833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 982/1968] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:13 2022] DELETE /identity/v3/users/0192aca24edb4293a0bbe47acdd906b1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:14.110056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 987/1969] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:13 2022] DELETE /identity/v3/users/4f2f09c3a8044bb18bc6b34c8641efab => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:14.504600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:14.654434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:14.656700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:14.685788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:14.687333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:14.737077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17f952-1017-4bb6-a10c-eb4f2c3d882a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:14.755244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c3da7-b615-4d18-bb84-8fff245c5575 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:14.760241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:14.760897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:14.761182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:14.761457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] PATH_INFO: `/v3/projects/61ead368719a49e7b56fb4b3d004e46e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:14.762613 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ead368719a49e7b56fb4b3d004e46e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:14.764517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:14.791766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:14.792409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:14.792683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:14.792957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] PATH_INFO: `/v3/projects/41dd4662f9bd4d9aafa230e92dafa85e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:14.794085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dd4662f9bd4d9aafa230e92dafa85e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:14.796082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:14.859054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ead368719a49e7b56fb4b3d004e46e', 'request_id': 'req-665c3aff-d292-43e0-a374-a049684b8ed9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.904459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dd4662f9bd4d9aafa230e92dafa85e', 'request_id': 'req-f81b9dff-38d7-4179-a467-2821639993c0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.909772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ead368719a49e7b56fb4b3d004e46e', 'request_id': 'req-665c3aff-d292-43e0-a374-a049684b8ed9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.910717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:14.911982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dd4662f9bd4d9aafa230e92dafa85e', 'request_id': 'req-f81b9dff-38d7-4179-a467-2821639993c0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.916256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ead368719a49e7b56fb4b3d004e46e', 'request_id': 'req-665c3aff-d292-43e0-a374-a049684b8ed9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.927398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] The token cache is being invalidate because project 61ead368719a49e7b56fb4b3d004e46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:14.927683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.928363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-665c3aff-d292-43e0-a374-a049684b8ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.929843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 988/1970] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:14 2022] DELETE /identity/v3/projects/61ead368719a49e7b56fb4b3d004e46e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:14.934579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36aec6-02cf-41e9-a273-fdb6825aad29 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:14.934865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36aec6-02cf-41e9-a273-fdb6825aad29 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:14.935142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36aec6-02cf-41e9-a273-fdb6825aad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:14.954957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6d36aec6-02cf-41e9-a273-fdb6825aad29 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:14.956448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6d36aec6-02cf-41e9-a273-fdb6825aad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:14.976531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dd4662f9bd4d9aafa230e92dafa85e', 'request_id': 'req-f81b9dff-38d7-4179-a467-2821639993c0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.988195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] The token cache is being invalidate because project 41dd4662f9bd4d9aafa230e92dafa85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:14.988484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.989114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:14.990641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 983/1971] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:14 2022] DELETE /identity/v3/projects/41dd4662f9bd4d9aafa230e92dafa85e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:14.994864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:14.995793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:15.015442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 989/1972] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:15.027398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.029077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.029077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.051474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:15.053245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:15.060096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b9dff-38d7-4179-a467-2821639993c0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.061741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 990/1973] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:15.065852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.067780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.068419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.068701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.068978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] PATH_INFO: `/v3/users/dba62e74621a4bb0ade54b0f8aa5d2ec` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.070101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba62e74621a4bb0ade54b0f8aa5d2ec)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.072114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.074901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a586f6c-9ae0-4fd3-ac33-38ce8121263e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.080055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.080694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.081231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.081508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.082183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:15.082522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.084127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.097825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 991/1974] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:30:15.100537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.110685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0b7e87-499e-4c1b-8713-aa03d456c051 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.115402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.115977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.116251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.116519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.117128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.119112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.232194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 992/1975] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:15.236022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.294636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba62e74621a4bb0ade54b0f8aa5d2ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:15.302530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba62e74621a4bb0ade54b0f8aa5d2ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:15.306157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58b2ce-1b0f-44d7-8f51-19118b3ecfa0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.311209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.312668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.312941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.313208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.313840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.315788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.365410 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba62e74621a4bb0ade54b0f8aa5d2ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:15.366041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba62e74621a4bb0ade54b0f8aa5d2ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:15.379073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba62e74621a4bb0ade54b0f8aa5d2ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:15.385140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 984/1976] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:14 2022] DELETE /identity/v3/users/dba62e74621a4bb0ade54b0f8aa5d2ec => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:15.406172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 993/1977] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:15.406881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.408323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:15.479826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598e2e6-a0f3-4a37-b2d6-0fa9dec952ba admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.485192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.485742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.486018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.486288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.486959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.488740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.498811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 985/1978] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:15.503546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.513491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff3f69-29bc-4043-af79-cd248818252b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.518086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.518715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.518991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.519263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.519905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.521488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2bab3a-e8aa-42be-bd3c-915c56550c6f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.531167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 994/1979] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:15.537849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.548241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396a95c-95d5-400b-a006-b84f0226458d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.553078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.553732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.554011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.554316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] PATH_INFO: `/v3/projects/e3d4cd5c215e4f39b73755cdfa8886bc/users/e158d5b286774b6f9c8ee2689006ebd7/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.569832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d4cd5c215e4f39b73755cdfa8886bc, user_id=e158d5b286774b6f9c8ee2689006ebd7, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.572710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.692116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 986/1980] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:15 2022] PUT /identity/v3/projects/e3d4cd5c215e4f39b73755cdfa8886bc/users/e158d5b286774b6f9c8ee2689006ebd7/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:15.703178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.703392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.703664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.734928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] MFA Rules not processed for user `e158d5b286774b6f9c8ee2689006ebd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:15.736293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:15.804620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 995/1981] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:15.809217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.869830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef9bf5-b742-45c7-9873-ee20aa8616d0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.876354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.876354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.876354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.876626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.877469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.878025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.879134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.879511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:15.894186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb56fc-11f4-4c23-b0ea-193230a4484b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:15.899619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:15.900265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:15.900546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:15.900819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] PATH_INFO: `/v3/projects/187485d1f0cb4842ba9f9188c216c883` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:15.901957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187485d1f0cb4842ba9f9188c216c883)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:15.904268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:15.910180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:15.972347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 987/1982] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:15.976383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:15.994770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:16.012502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187485d1f0cb4842ba9f9188c216c883', 'request_id': 'req-8b682497-0623-4435-af58-0bbcabc3fda5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:16.037462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7844334-c24d-480e-920e-07e5eb0055b5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.093474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.094087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.094367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.094669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.095297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.097218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.104264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187485d1f0cb4842ba9f9188c216c883', 'request_id': 'req-8b682497-0623-4435-af58-0bbcabc3fda5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:16.110940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187485d1f0cb4842ba9f9188c216c883', 'request_id': 'req-8b682497-0623-4435-af58-0bbcabc3fda5'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:16.122211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] The token cache is being invalidate because project 187485d1f0cb4842ba9f9188c216c883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:16.122558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:16.123207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8b682497-0623-4435-af58-0bbcabc3fda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:16.124707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 996/1983] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:15 2022] DELETE /identity/v3/projects/187485d1f0cb4842ba9f9188c216c883 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:16.129343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7808-02f8-4125-999a-f3e16ce0227b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.129624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7808-02f8-4125-999a-f3e16ce0227b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.129898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b7808-02f8-4125-999a-f3e16ce0227b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.148896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9d9b7808-02f8-4125-999a-f3e16ce0227b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:16.150097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d9b7808-02f8-4125-999a-f3e16ce0227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.158982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 997/1984] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:16.167387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.167673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.167951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.187472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:16.188789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.200970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 998/1985] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:16.203183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 988/1986] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:15 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:16.206134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.208532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.210252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.215848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9794ab-7c90-40d6-9d1f-45e44cd5b593 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.221341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.221943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.222222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.222583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.222878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f195b3e-35aa-468b-810b-07c50033a4be admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.223183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:16.223486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.225084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.227588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.228472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.228686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.228969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.229614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.231246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.239365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 999/1987] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:30:16.243324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 989/1988] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:16.244329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.246280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.256790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd774d01-d8a2-466c-84d2-8aef33731741 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.257290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c16fa-1c38-416e-9d77-24718573735b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.261803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.262204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.262496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.262782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.264413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.264510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] PATH_INFO: `/v3/projects/175cd93f68cb4ea68a1534146a70187c/users/c18e099b74db458db20d31f834bf0972/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.265161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.265161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.265161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.266508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.279636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175cd93f68cb4ea68a1534146a70187c, user_id=c18e099b74db458db20d31f834bf0972, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.283053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.348333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1000/1989] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:16.352726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.362200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 990/1990] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:16 2022] PUT /identity/v3/projects/175cd93f68cb4ea68a1534146a70187c/users/c18e099b74db458db20d31f834bf0972/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:16.366227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.424013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d3718-5ea8-45bb-b72e-e3c7b09a5f5a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.428999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.429542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.429649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd90d1-19ba-43e9-93df-dbfe55627e31 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.429834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.430116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.430789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.432708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.434390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.435051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.435622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.436129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.437240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.439316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.449609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 991/1991] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:16.454364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.464210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554843a-94b0-4490-b54f-b8dee61973b8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.469416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.470041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.470322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.470652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] PATH_INFO: `/v3/projects/175cd93f68cb4ea68a1534146a70187c/users/c18e099b74db458db20d31f834bf0972/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.473986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175cd93f68cb4ea68a1534146a70187c, user_id=c18e099b74db458db20d31f834bf0972, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.476700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d64145-8b45-48af-b280-f3edefa34874 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.489042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 992/1992] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:16 2022] PUT /identity/v3/projects/175cd93f68cb4ea68a1534146a70187c/users/c18e099b74db458db20d31f834bf0972/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:16.499482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.499768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.500039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.528137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1001/1993] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:16.532527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] MFA Rules not processed for user `c18e099b74db458db20d31f834bf0972`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:16.533209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.535030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.551379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ca8bd-2fcb-489c-adde-cdd30049be9e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.620871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 993/1994] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:16.636211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.642636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.642636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.642636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.642636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.642636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.650469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.651919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.660547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1002/1995] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:16.664536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.676449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8274-1d80-4c88-afbc-e0abc084297c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.682685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.683622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.697320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.697320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.697320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.697320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.699349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1003/1996] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:16.703852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.714672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6a064-fd47-4604-b500-786b7dc8bbe8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.718429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fcfce-1d20-4108-bf12-a913d252ca45 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:16.721062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.721725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.722004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.722286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] PATH_INFO: `/v3/projects/1297d0249cf34d7290e867124be62377/users/881062ec3efc4445a43e1ff772b09153/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.722443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:16.723014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.723292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.723563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.725442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.739186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1297d0249cf34d7290e867124be62377, user_id=881062ec3efc4445a43e1ff772b09153, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.742012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.784885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:16.786336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3029955-7765-4b9f-9ba9-bbae12a92787 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:16.795202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 994/1997] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:16.867891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1004/1998] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:16 2022] PUT /identity/v3/projects/1297d0249cf34d7290e867124be62377/users/881062ec3efc4445a43e1ff772b09153/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:16.873409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1acfc0-cf6e-47ea-9f05-fdefb8a5597a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:16.873701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1acfc0-cf6e-47ea-9f05-fdefb8a5597a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:16.874517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1acfc0-cf6e-47ea-9f05-fdefb8a5597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:16.885281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:16.896434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2b1acfc0-cf6e-47ea-9f05-fdefb8a5597a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:16.897638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2b1acfc0-cf6e-47ea-9f05-fdefb8a5597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:16.949989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:16.956360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 995/1999] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:16.965874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:16.967550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:17.022337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c16f15-14ed-4030-a1c3-b0eeca859f38 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:17.026332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:17.026969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:17.027247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:17.027518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:17.036655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:17.038196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:17.047989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1005/2000] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:18.059151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:18.122357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:18.446708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c7c5-cf9a-4dbf-9965-c6b53d3f3d09 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:18.446995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c7c5-cf9a-4dbf-9965-c6b53d3f3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:18.447294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8c7c5-cf9a-4dbf-9965-c6b53d3f3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:18.475448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-edd8c7c5-cf9a-4dbf-9965-c6b53d3f3d09 None None] MFA Rules not processed for user `881062ec3efc4445a43e1ff772b09153`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:18.477865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-edd8c7c5-cf9a-4dbf-9965-c6b53d3f3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:18.543936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:18.547712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 996/2001] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:18.562563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:18.617572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59208ea4-9122-47ab-86ed-914a9a5ac282 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:18.621185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:18.621792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:18.622082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:18.622342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:18.631037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:18.632479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:18.643038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1006/2002] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:19.416879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:19.549847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:20.425244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c957d-b0ce-4fd1-9e0a-2859f12cbcb1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:20.425244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c957d-b0ce-4fd1-9e0a-2859f12cbcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:20.425244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c957d-b0ce-4fd1-9e0a-2859f12cbcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:20.442759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-385c957d-b0ce-4fd1-9e0a-2859f12cbcb1 None None] MFA Rules not processed for user `881062ec3efc4445a43e1ff772b09153`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:20.443955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-385c957d-b0ce-4fd1-9e0a-2859f12cbcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:20.451494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 997/2003] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:20.584629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:21.029922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:21.056118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89cad5-b0ba-4968-a6bc-cdcd1b6ae6e1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:21.064920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:21.065716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:21.066157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:21.066625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] PATH_INFO: `/v3/users/e158d5b286774b6f9c8ee2689006ebd7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:21.069351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e158d5b286774b6f9c8ee2689006ebd7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:21.072476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:21.378041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e158d5b286774b6f9c8ee2689006ebd7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:21.398386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e158d5b286774b6f9c8ee2689006ebd7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:21.413279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e158d5b286774b6f9c8ee2689006ebd7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:21.414642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e158d5b286774b6f9c8ee2689006ebd7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:21.416209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e158d5b286774b6f9c8ee2689006ebd7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:21.458782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:21.532802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1007/2004] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:21 2022] DELETE /identity/v3/users/e158d5b286774b6f9c8ee2689006ebd7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:21.537309 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:21.537786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:21.537880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:21.567413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:21.568632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:21.643312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 998/2005] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:21.665935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:21.667424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:21.745569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7edb8-24ab-4d7a-a2e9-171cc9c66587 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:21.749964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:21.750624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:21.750905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:21.751310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:21.761373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:21.763101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:21.774942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1008/2006] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:22.378122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:22.379593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:22.394288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b17de-7639-48e1-bbed-341dea665a84 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:22.398912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:22.399551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:22.399880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:22.400104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] PATH_INFO: `/v3/projects/e3d4cd5c215e4f39b73755cdfa8886bc` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:22.401328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d4cd5c215e4f39b73755cdfa8886bc)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:22.403205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:22.549930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d4cd5c215e4f39b73755cdfa8886bc', 'request_id': 'req-0a5a3174-dc70-41e4-ab13-d7a9db80a515'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.556757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d4cd5c215e4f39b73755cdfa8886bc', 'request_id': 'req-0a5a3174-dc70-41e4-ab13-d7a9db80a515'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.652070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d4cd5c215e4f39b73755cdfa8886bc', 'request_id': 'req-0a5a3174-dc70-41e4-ab13-d7a9db80a515'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.665904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] The token cache is being invalidate because project e3d4cd5c215e4f39b73755cdfa8886bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:22.666257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.667118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.668835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 999/2007] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:22 2022] DELETE /identity/v3/projects/e3d4cd5c215e4f39b73755cdfa8886bc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:22.673004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:22.683704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8f60a-06a5-47d9-8e57-c17458fb4fd5 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:22.756413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:22.757093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:22.757668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:22.757951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] PATH_INFO: `/v3/users/c18e099b74db458db20d31f834bf0972` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:22.759194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18e099b74db458db20d31f834bf0972)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:22.761565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:22.929399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:22.931353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:22.967480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18e099b74db458db20d31f834bf0972'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.984984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18e099b74db458db20d31f834bf0972'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.994272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18e099b74db458db20d31f834bf0972'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.995206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18e099b74db458db20d31f834bf0972'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:22.996306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18e099b74db458db20d31f834bf0972'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.012712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a3174-dc70-41e4-ab13-d7a9db80a515 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:23.016571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:23.017449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:23.017911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:23.019337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:23.020091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:23.022087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:23.090924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:23.092602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1009/2008] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:22 2022] DELETE /identity/v3/users/c18e099b74db458db20d31f834bf0972 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:23.095472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299de44c-ee45-412d-8715-280169e98b54 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:23.110230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1000/2009] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:23.622928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:23.625236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:23.716741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85dda5-63f2-4d7e-a4ee-01b7d5346caf admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:23.722048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:23.722878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:23.723325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:23.723739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] PATH_INFO: `/v3/projects/175cd93f68cb4ea68a1534146a70187c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:23.725262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175cd93f68cb4ea68a1534146a70187c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:23.728140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:23.836639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175cd93f68cb4ea68a1534146a70187c', 'request_id': 'req-0844b809-f666-4cd0-acb7-4b45ecd4c595'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.858744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:23.921089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175cd93f68cb4ea68a1534146a70187c', 'request_id': 'req-0844b809-f666-4cd0-acb7-4b45ecd4c595'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.927359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175cd93f68cb4ea68a1534146a70187c', 'request_id': 'req-0844b809-f666-4cd0-acb7-4b45ecd4c595'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.938999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] The token cache is being invalidate because project 175cd93f68cb4ea68a1534146a70187c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:23.939287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.939930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0844b809-f666-4cd0-acb7-4b45ecd4c595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:23.943074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1010/2010] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:23 2022] DELETE /identity/v3/projects/175cd93f68cb4ea68a1534146a70187c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:23.947847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:23.948291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:23.948428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:23.969847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:23.971018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:24.040229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1001/2011] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:24.048977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.049492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.049932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.071748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:24.073250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:24.082270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1011/2012] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:24.087169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.098078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d42ac-d89c-4765-82cc-62c9ba454bcf None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.105708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.108630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.109761 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.110566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.111474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:24.112095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.114216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.139749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1002/2013] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:24.141005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.152495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef188b-542e-4384-96af-982b49fbb482 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.165271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.167739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.168142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.168618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.169477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.173014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.286811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.288505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1012/2014] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:24 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:24.300772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77850ae-9a69-4921-932a-880d329bd251 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.364856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.365468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.365749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.366022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.366703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.368640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.506041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.513066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1003/2015] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:24 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:24.520970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28d0cc-4764-4207-ac8b-9c61a4e2e53b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.526765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.531018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.531018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.531018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.531018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.531018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.544517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1013/2016] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:24.548620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.559358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f5b24-dbc7-4267-b330-539805356c73 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.579862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.580768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.581072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.581349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.581991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.583706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b171f-0e63-483f-ad2a-9d299fb572e2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.605242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1004/2017] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:24.609798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.620229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3ef3e-4c22-4123-a1ea-471bd2574a7b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:24.625744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:24.626822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.627100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.627376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] PATH_INFO: `/v3/projects/690ee9d980b44445941cb08d4f84904c/users/7ed9ce5ab1c34b9797aea3d851baf8a4/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.646540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690ee9d980b44445941cb08d4f84904c, user_id=7ed9ce5ab1c34b9797aea3d851baf8a4, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:24.649295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:24.797474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1014/2018] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:24 2022] PUT /identity/v3/projects/690ee9d980b44445941cb08d4f84904c/users/7ed9ce5ab1c34b9797aea3d851baf8a4/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:24.810048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:24.810334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:24.812780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:24.840069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] MFA Rules not processed for user `7ed9ce5ab1c34b9797aea3d851baf8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:24.841288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:24.921381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1005/2019] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:24.932652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:24.934429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:25.025796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b0023-acbd-41d1-8163-af8fdf7362b1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:25.029852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:25.030561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:25.030875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:25.031153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:25.048827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:25.050716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e20f3-8ebc-400d-87e4-e1215208eff2 service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:25.062939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1015/2020] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:25.168334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:25.256744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:26.876499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:27.859119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:28.556635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:28.611913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:28.613390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:28.724499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d6ed9-026b-4605-91b6-bccac2f815e7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:28.729500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:28.730155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:28.730629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:28.731000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] PATH_INFO: `/v3/users/c5527be4561b4ff3868c1b3b9a81cbcc` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:28.732133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5527be4561b4ff3868c1b3b9a81cbcc)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:28.735575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:28.969716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5527be4561b4ff3868c1b3b9a81cbcc'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:28.977343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5527be4561b4ff3868c1b3b9a81cbcc'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:29.078847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5527be4561b4ff3868c1b3b9a81cbcc'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:29.079589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5527be4561b4ff3868c1b3b9a81cbcc'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:29.093867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5527be4561b4ff3868c1b3b9a81cbcc'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:29.099806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1006/2021] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:28 2022] DELETE /identity/v3/users/c5527be4561b4ff3868c1b3b9a81cbcc => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:29.104249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:29.109413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:29.109413 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:29.127134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:29.128342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:29.209231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1016/2022] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:29.220279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:29.223292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:29.310482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2235b36-5272-4784-ba7d-4e455391d80f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:29.325552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:29.326446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:29.326971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:29.327146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:29.342117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:29.343981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:29.356894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1007/2023] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:29.957733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:29.959237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:29.972304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc585-a8be-4db2-aa16-4b4cb3ee4d31 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:29.981998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:29.983231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:29.983475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:29.983750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] PATH_INFO: `/v3/projects/9eb8e7ae881841c0afe23346de9e0761` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:29.984905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb8e7ae881841c0afe23346de9e0761)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:29.986884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:29.996862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:30.093063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:30.100206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb8e7ae881841c0afe23346de9e0761', 'request_id': 'req-67a3160d-1ec2-4904-84ee-bf2e030051cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.183698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb8e7ae881841c0afe23346de9e0761', 'request_id': 'req-67a3160d-1ec2-4904-84ee-bf2e030051cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.189993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb8e7ae881841c0afe23346de9e0761', 'request_id': 'req-67a3160d-1ec2-4904-84ee-bf2e030051cf'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.204006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] The token cache is being invalidate because project 9eb8e7ae881841c0afe23346de9e0761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:30.204304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.205030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.206872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1017/2024] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:29 2022] DELETE /identity/v3/projects/9eb8e7ae881841c0afe23346de9e0761 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:30.211335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:30.221973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a390c-57b1-4dc3-9c2b-884a1779b5ac service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:30.281289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:30.281952 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:30.282301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:30.282668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] PATH_INFO: `/v3/users/ca498d5a6fb74a89b71ea7756a6aa4ae` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:30.283837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca498d5a6fb74a89b71ea7756a6aa4ae)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:30.285847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:30.453668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca498d5a6fb74a89b71ea7756a6aa4ae'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.461011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca498d5a6fb74a89b71ea7756a6aa4ae'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.553418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca498d5a6fb74a89b71ea7756a6aa4ae'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.555189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca498d5a6fb74a89b71ea7756a6aa4ae'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.587196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca498d5a6fb74a89b71ea7756a6aa4ae'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:30.596900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1008/2025] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:30 2022] DELETE /identity/v3/users/ca498d5a6fb74a89b71ea7756a6aa4ae => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:31.197313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:31.207027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:31.219079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3160d-1ec2-4904-84ee-bf2e030051cf admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:31.269460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:31.278262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:31.279049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:31.279727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:31.281424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] PATH_INFO: `/v3/projects/92ebdbeaf4b2414d9c0f9207e182bcdb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:31.282761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ebdbeaf4b2414d9c0f9207e182bcdb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:31.284923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:31.331935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:31.408052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ebdbeaf4b2414d9c0f9207e182bcdb', 'request_id': 'req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.480516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ebdbeaf4b2414d9c0f9207e182bcdb', 'request_id': 'req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.493438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ebdbeaf4b2414d9c0f9207e182bcdb', 'request_id': 'req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.501418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] The token cache is being invalidate because project 92ebdbeaf4b2414d9c0f9207e182bcdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:31.501707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.502619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.504332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1018/2026] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:31 2022] DELETE /identity/v3/projects/92ebdbeaf4b2414d9c0f9207e182bcdb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:31.508278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:31.509803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:31.526532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaff6a4-c9e4-45d4-9aea-193d6c497d9e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:31.590813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:31.591462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:31.592339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:31.592985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] PATH_INFO: `/v3/users/ee54019c09cb44f2b986611234abdf9c` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:31.594300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee54019c09cb44f2b986611234abdf9c)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:31.596708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:31.814552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee54019c09cb44f2b986611234abdf9c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.822008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee54019c09cb44f2b986611234abdf9c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.910184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee54019c09cb44f2b986611234abdf9c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.910857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee54019c09cb44f2b986611234abdf9c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.931962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-750deb95-5efd-46ab-add4-4e2cb7e5190f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee54019c09cb44f2b986611234abdf9c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:31.954081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1009/2027] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:31 2022] DELETE /identity/v3/users/ee54019c09cb44f2b986611234abdf9c => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:32.488084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:32.490828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:32.492148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:32.509277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81cc21-ab2a-4c71-b3ff-9eb953a894b1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:32.572817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:32.573460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:32.573759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:32.574036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] PATH_INFO: `/v3/projects/36d150afa2084c7b9b604e533ec486e8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:32.575221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d150afa2084c7b9b604e533ec486e8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:32.577518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:32.712076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d150afa2084c7b9b604e533ec486e8', 'request_id': 'req-0b7e742c-a9dc-44e2-8877-426a8011d897'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:32.777407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d150afa2084c7b9b604e533ec486e8', 'request_id': 'req-0b7e742c-a9dc-44e2-8877-426a8011d897'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:32.782060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:32.787063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d150afa2084c7b9b604e533ec486e8', 'request_id': 'req-0b7e742c-a9dc-44e2-8877-426a8011d897'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:32.801507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] The token cache is being invalidate because project 36d150afa2084c7b9b604e533ec486e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:32.801507 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:32.802005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0b7e742c-a9dc-44e2-8877-426a8011d897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:32.803773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1019/2028] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:32 2022] DELETE /identity/v3/projects/36d150afa2084c7b9b604e533ec486e8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:32.809492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:32.809782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:32.810170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:32.831807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:32.833886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:32.899346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1010/2029] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:32.908215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:32.908526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:32.908801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:32.930529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:32.931446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:32.947072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1020/2030] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:32.951702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:32.962974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb53b8-fb53-48fe-be3f-3886e71c6b3f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:32.969594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:32.970462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:32.970943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:32.971351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:32.972218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:32.972819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:32.975067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:32.990589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1011/2031] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:32.994760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.017083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953d80-c64a-46a6-a299-fc1353b8ada1 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.022707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.023671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.024291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.024734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.025731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.028220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.122452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1021/2032] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:32 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:33.127435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.140942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5613-68d9-4c89-8ed5-c58b0f2e3209 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.209230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.209934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.210277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.210778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.212164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.214290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.347327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1012/2033] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:33 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:33.348633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.359029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e0e4c-f75a-498d-bcfe-8c69342749ec admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.368384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.368384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.368384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.368384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.368792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.368792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.383332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1022/2034] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:33.384591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.396570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92974c-e1d2-4000-b711-e51931b395d5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.402037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.402896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.403228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.403500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.404208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.406243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58deca1-41c3-4cac-bcea-c2ab11597191 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.419205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1013/2035] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:33.423382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.434925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d7600-868c-422c-9d55-3854e62f2063 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.440405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.441472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.441821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.442195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] PATH_INFO: `/v3/projects/4bf6b4394c2d4fc5bac3f426f3ee0b19/users/0691ba899d0e404aa04465921d7e49e4/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.460487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf6b4394c2d4fc5bac3f426f3ee0b19, user_id=0691ba899d0e404aa04465921d7e49e4, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.464075 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.578667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1023/2036] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:33 2022] PUT /identity/v3/projects/4bf6b4394c2d4fc5bac3f426f3ee0b19/users/0691ba899d0e404aa04465921d7e49e4/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:33.580603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.580892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.581170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.614037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] MFA Rules not processed for user `0691ba899d0e404aa04465921d7e49e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:33.615308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:33.719116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1014/2037] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:33.723674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.792891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735105e4-c368-41df-a95a-076ba80d76f2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.799236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.799962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.800228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.800496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.801490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.803659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:33.887603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1024/2038] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:33 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:33.892208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:33.904177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0784eb-9e80-4224-94c3-06317c19863e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:33.961864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:33.962510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:33.962791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:33.963063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:33.963708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:33.965698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.077377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1015/2039] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:33 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:34.082423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.094228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547754ac-2013-4b07-b767-fd4d4da50cd1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:34.100429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:34.101057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.101334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.101641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.102440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:34.104199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.131188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1025/2040] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:34.135107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.155581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f5492-3d7c-4f88-be9a-fc558aac4b04 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:34.155581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:34.155581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.155581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.156143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] PATH_INFO: `/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2/users/da85961b9d564c889b8fa0c737e97115/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.183109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf94fc270821436dbc34b7ee3a0af3a2, user_id=da85961b9d564c889b8fa0c737e97115, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:34.186860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.273926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1016/2041] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:34 2022] PUT /identity/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2/users/da85961b9d564c889b8fa0c737e97115/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:34.278035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.287883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940293d-2b74-4aef-a933-2e9f57dd04df admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:34.359820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:34.382003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:34.382660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.382991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.383270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.383959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:34.385590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa7117-0645-4433-a95e-ce322216d60f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.400709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1026/2042] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:34.410062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.424708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09690842-86b9-41b0-bc6b-3e2dd3f1ce3c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:34.431388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:34.431388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.431388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.432085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] PATH_INFO: `/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2/users/da85961b9d564c889b8fa0c737e97115/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.434337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf94fc270821436dbc34b7ee3a0af3a2, user_id=da85961b9d564c889b8fa0c737e97115, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:34.437806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.450331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1017/2043] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:34 2022] PUT /identity/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2/users/da85961b9d564c889b8fa0c737e97115/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:34.462020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.462315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.462620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.495925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] MFA Rules not processed for user `da85961b9d564c889b8fa0c737e97115`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:34.500191 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:34.597847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1027/2044] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:34.610077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.611760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:34.670380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cca371-1616-4cf4-a78c-82bf41116eb8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:34.674223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:34.674871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:34.675152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:34.675423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:34.685252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:34.686637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:34.695281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:34.699485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1018/2045] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:34.973641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:34.996995 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc02f79-3e37-417f-a433-49c9b4331de5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.018215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.018984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.019367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.019724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.090622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.092759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:35.102677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1028/2046] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:35.800682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:35.802244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:35.867449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:35.868965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f16688-6e08-4d33-985c-499c3c447ce2 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:35.874367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:35.874862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:35.875148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:35.875422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] PATH_INFO: `/v3/users/55c9d3a3577746a4ad7406fe4396eaee` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:35.876688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c9d3a3577746a4ad7406fe4396eaee)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:35.878760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.082559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c9d3a3577746a4ad7406fe4396eaee'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.089367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c9d3a3577746a4ad7406fe4396eaee'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.176542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c9d3a3577746a4ad7406fe4396eaee'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.178324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:36.184061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c9d3a3577746a4ad7406fe4396eaee'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.191699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7907b235-e3ba-44d3-9368-ebadab94c0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c9d3a3577746a4ad7406fe4396eaee'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.197726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1019/2047] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:35 2022] DELETE /identity/v3/users/55c9d3a3577746a4ad7406fe4396eaee => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:36.464621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:36.478334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6afbaa-300a-4ef5-8c79-300cbd6649f1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:36.601062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:36.601919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.602496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.602961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] PATH_INFO: `/v3/projects/cc23f8d119514ad595637967b626e87d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.604495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc23f8d119514ad595637967b626e87d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:36.606626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:36.805729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc23f8d119514ad595637967b626e87d', 'request_id': 'req-9c4cc256-37a2-4109-950d-c299e3f169ad'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.885911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc23f8d119514ad595637967b626e87d', 'request_id': 'req-9c4cc256-37a2-4109-950d-c299e3f169ad'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.892699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc23f8d119514ad595637967b626e87d', 'request_id': 'req-9c4cc256-37a2-4109-950d-c299e3f169ad'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.904547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] The token cache is being invalidate because project cc23f8d119514ad595637967b626e87d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:36.904839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.905496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c4cc256-37a2-4109-950d-c299e3f169ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:36.907234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1029/2048] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:36 2022] DELETE /identity/v3/projects/cc23f8d119514ad595637967b626e87d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:36.914185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:36.914185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:36.914185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:36.939954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:36.941832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:37.013692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1020/2049] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:37.022015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.022195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.026479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.044281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:37.050854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:37.054269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1030/2050] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:37.058292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.069544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d14a-fdb0-462e-9404-d032e96d65d8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.074650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.079473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.092450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1021/2051] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:37.096700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.106686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64076e9-cc44-47ff-af69-dad45476d7ed None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.110616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.117196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.117923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.118296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.118671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.119504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.121659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.188072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.229444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1031/2052] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:37.233976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.244999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c49db3-654a-4513-b559-a5dd12d585c4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.317651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.318259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.318741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.319356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.324427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.324427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.428227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1022/2053] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:37.432407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.443376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b649aa-6872-47b6-82f4-c4447140b2de admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.447981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.448623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.451433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.451433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.451433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.452008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.461986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1032/2054] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.465991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.477455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576f037-9239-4cf4-a2a7-c9a27463453e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.482224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.482924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.483271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.483616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.484326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.486065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb0fd6-9271-4552-b3ed-180a135ece81 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.496767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1023/2055] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:37.501082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.511963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c0291a-b9fd-4614-9444-6e5f304c5f27 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.517113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.520554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.520554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.520554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] PATH_INFO: `/v3/projects/cad65f0671c74411a13bc1d12229e8f9/users/bb6e38cb0019468ba7bb0962af5b5445/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.533254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad65f0671c74411a13bc1d12229e8f9, user_id=bb6e38cb0019468ba7bb0962af5b5445, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.536006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.606827 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1033/2056] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:37 2022] PUT /identity/v3/projects/cad65f0671c74411a13bc1d12229e8f9/users/bb6e38cb0019468ba7bb0962af5b5445/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:37.618045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.618326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.618636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.658241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] MFA Rules not processed for user `bb6e38cb0019468ba7bb0962af5b5445`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:37.659491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:37.732978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1024/2057] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:37.744344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:37.746140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:37.772908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.805951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:37.825615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3c965-a39b-4c57-a7e9-858b0856433c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:37.832056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:37.832660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:37.832935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:37.833200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:37.840790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:37.842773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf467ea8-cd2e-4f03-870a-a048bcea3c93 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:37.851834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1034/2058] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:38.092059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:38.240413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:38.782165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:38.787699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:38.896661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cfa13-5583-4d81-a4e0-67e05ca38ee4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:38.906054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:38.906850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:38.907135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:38.908722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] PATH_INFO: `/v3/users/58ff9c1d5b4b44b1815fc9252df81867` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:38.909938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ff9c1d5b4b44b1815fc9252df81867)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:38.912081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:38.999477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.097265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.174213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ff9c1d5b4b44b1815fc9252df81867'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:39.181348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ff9c1d5b4b44b1815fc9252df81867'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:39.290092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ff9c1d5b4b44b1815fc9252df81867'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:39.290856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ff9c1d5b4b44b1815fc9252df81867'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:39.306525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ff9c1d5b4b44b1815fc9252df81867'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:39.316655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1025/2059] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:38 2022] DELETE /identity/v3/users/58ff9c1d5b4b44b1815fc9252df81867 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:39.321472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.321737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.322184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.348896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:39.350480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:39.405128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:39.459041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1035/2060] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:39.469806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:39.471331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:39.535210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e5ed9-31fb-40fa-823a-aaa109a1331a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:39.538778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:39.539378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:39.539651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:39.539920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:39.549864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:39.551355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:39.561254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1026/2061] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:40.160038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.165919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:40.187735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5d655-691a-4250-be8c-374d7bf9f68d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.190941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.191629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.192143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.192581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] PATH_INFO: `/v3/projects/ea7ae437f48041f08164153e7a4945ef` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.194632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7ae437f48041f08164153e7a4945ef)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.196612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.358250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7ae437f48041f08164153e7a4945ef', 'request_id': 'req-1d7ebd65-6c86-4fd2-b437-c60174067728'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.444703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:40.472781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7ae437f48041f08164153e7a4945ef', 'request_id': 'req-1d7ebd65-6c86-4fd2-b437-c60174067728'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.479676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7ae437f48041f08164153e7a4945ef', 'request_id': 'req-1d7ebd65-6c86-4fd2-b437-c60174067728'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.514116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] The token cache is being invalidate because project ea7ae437f48041f08164153e7a4945ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:40.514275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.515439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.517467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1036/2062] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:40 2022] DELETE /identity/v3/projects/ea7ae437f48041f08164153e7a4945ef => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:40.521254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:40.533897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468a10c-3a98-46a1-80bf-2446f2543f40 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:40.600675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:40.601513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:40.601802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:40.602250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] PATH_INFO: `/v3/users/f1555a881b76470984ee3a7203e92f15` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:40.618499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1555a881b76470984ee3a7203e92f15)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:40.618499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:40.626293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:40.893964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1555a881b76470984ee3a7203e92f15'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:40.905936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1555a881b76470984ee3a7203e92f15'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:41.028276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1555a881b76470984ee3a7203e92f15'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:41.029454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1555a881b76470984ee3a7203e92f15'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:41.042734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8ff93db5-5c40-41d3-bf43-865bd79f4abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1555a881b76470984ee3a7203e92f15'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:41.049440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1027/2063] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:40 2022] DELETE /identity/v3/users/f1555a881b76470984ee3a7203e92f15 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:41.175582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:41.251911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:41.768206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:41.769474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:41.788945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ebd65-6c86-4fd2-b437-c60174067728 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:41.861077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:41.862080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:41.862389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:41.862742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] PATH_INFO: `/v3/projects/db6e2ab5ef324a2990eea647107fe6f2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:41.864095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6e2ab5ef324a2990eea647107fe6f2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:41.866033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:41.973024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6e2ab5ef324a2990eea647107fe6f2', 'request_id': 'req-3d0b801e-aea1-436c-afee-b7532e1f6683'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.061434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6e2ab5ef324a2990eea647107fe6f2', 'request_id': 'req-3d0b801e-aea1-436c-afee-b7532e1f6683'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.068841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6e2ab5ef324a2990eea647107fe6f2', 'request_id': 'req-3d0b801e-aea1-436c-afee-b7532e1f6683'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.081437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] The token cache is being invalidate because project db6e2ab5ef324a2990eea647107fe6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:42.081774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.082489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3d0b801e-aea1-436c-afee-b7532e1f6683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:42.084123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1037/2064] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:41 2022] DELETE /identity/v3/projects/db6e2ab5ef324a2990eea647107fe6f2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:42.089058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e22c-833f-426c-be62-ea4784867708 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.089586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e22c-833f-426c-be62-ea4784867708 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.089586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e22c-833f-426c-be62-ea4784867708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.117746 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7874e22c-833f-426c-be62-ea4784867708 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:42.119004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7874e22c-833f-426c-be62-ea4784867708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:42.148737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.202461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1028/2065] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:42.207397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.207972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.208253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.218724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.232154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:42.233603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:42.242805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1038/2066] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:42.247128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874e22c-833f-426c-be62-ea4784867708 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.258618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874e22c-833f-426c-be62-ea4784867708 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.263880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.264547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.264858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.265097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.265747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:42.266072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.267730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.280959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1029/2067] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:42.284481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.294128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d2709-e112-4b51-807e-54050f7e8d29 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.300594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.300594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.300801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.301051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.301685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.303738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.450931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1039/2068] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:42.453707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.468344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9832f-b395-46b6-8ea3-35f8634d23a2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.529554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.530156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.530456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.530730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.531368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.533325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.664936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1030/2069] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:42.669725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.681183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74eef27-3c69-46da-817b-5a274472a4c7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.687201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.687909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.688244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.688580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.689291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.691154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.701367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1040/2070] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:42.705797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.715975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c5584-578b-43bf-be51-37abea013b2a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.720788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.721383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.721654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.721919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.722588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.724346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef332411-ae6d-4946-bf6b-4ac350c4e404 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.736384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1031/2071] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:42.740484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:42.754745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bcf88-1b4d-4ffe-abce-737801aacede admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:42.758934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.760189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:42.761085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.761430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.761792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] PATH_INFO: `/v3/projects/38b7ecda83404572a25c0598c322f2e8/users/1fc72a80660240df9e395e43c8fa3dbd/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.781235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b7ecda83404572a25c0598c322f2e8, user_id=1fc72a80660240df9e395e43c8fa3dbd, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:42.790648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:42.861432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1041/2072] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:42 2022] PUT /identity/v3/projects/38b7ecda83404572a25c0598c322f2e8/users/1fc72a80660240df9e395e43c8fa3dbd/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:42.869945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:42.874717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:42.875001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:42.875273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:42.904163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] MFA Rules not processed for user `1fc72a80660240df9e395e43c8fa3dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:42.905317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:42.972029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1032/2073] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:42.976504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.029741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.040442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2313049-e74e-43a1-8c92-9bd79cb0fbc5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.046106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.056492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.056492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.056492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.056492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.056492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.080446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.121561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1042/2074] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:42 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:43.125806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.137511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e96a-a735-4826-9d5e-33c0dc6431e0 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.195790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.196323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.196607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.196873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.197531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.199517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.332533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1033/2075] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:43 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:43.336500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.352344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14497a11-f104-4aef-a439-11291f071092 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.357458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.358167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.358648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.358824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.359447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.361381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.373825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1043/2076] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:43.378803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.390250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a91953-d6ba-4070-9f3c-b53d507f485e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.396589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.397373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.397651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.397918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] PATH_INFO: `/v3/projects/78fd50775c7f4bd0a058e102ef23c26b/users/f3dab808e10a4acbabf5225c986c5ac8/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.414124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fd50775c7f4bd0a058e102ef23c26b, user_id=f3dab808e10a4acbabf5225c986c5ac8, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.416869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.499036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1034/2077] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:43 2022] PUT /identity/v3/projects/78fd50775c7f4bd0a058e102ef23c26b/users/f3dab808e10a4acbabf5225c986c5ac8/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:43.503268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.516777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb977d28-d3f1-46d0-b121-86ef166f4100 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.587815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.594167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.594920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.595288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.595440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.596088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.597666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b56af-e537-47cb-b210-8dee744d08e2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.608528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1044/2078] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:43.612567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.624141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9ec1f-4308-4f03-9d29-679eb939898f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.628630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.631909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.632600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.632879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.633623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] PATH_INFO: `/v3/projects/78fd50775c7f4bd0a058e102ef23c26b/users/f3dab808e10a4acbabf5225c986c5ac8/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.636537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fd50775c7f4bd0a058e102ef23c26b, user_id=f3dab808e10a4acbabf5225c986c5ac8, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.644061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.661779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1035/2079] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:43 2022] PUT /identity/v3/projects/78fd50775c7f4bd0a058e102ef23c26b/users/f3dab808e10a4acbabf5225c986c5ac8/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:43.669876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.669876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.669876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.701913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] MFA Rules not processed for user `f3dab808e10a4acbabf5225c986c5ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:43.701913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:43.776322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1045/2080] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:43.787526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:43.789282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:43.823104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.849980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:43.869894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45446afb-c6f6-4093-b02b-4b3ae2f3aafe admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:43.875552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:43.876438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:43.876885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:43.877260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:43.893434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:43.900470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:43.905754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1036/2081] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:44.194458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:44.575972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:45.117150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.125314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:45.275268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063615a-20d4-4720-8604-7fd76b04a3b6 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.280812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.281858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.282198 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.282590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] PATH_INFO: `/v3/users/1fc72a80660240df9e395e43c8fa3dbd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.284095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc72a80660240df9e395e43c8fa3dbd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.286271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:45.367873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.369272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:45.378132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:45.385396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147849-5df9-4829-bfa2-3cd2a10431dc service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:45.390710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:45.391347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.391631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.391904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] PATH_INFO: `/v3/users/0691ba899d0e404aa04465921d7e49e4` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.393035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0691ba899d0e404aa04465921d7e49e4)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:45.395027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:45.608196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc72a80660240df9e395e43c8fa3dbd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.622955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc72a80660240df9e395e43c8fa3dbd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.631820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc72a80660240df9e395e43c8fa3dbd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.635077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc72a80660240df9e395e43c8fa3dbd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.640157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f83bba31-909c-42d3-bba6-d88b4625e18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc72a80660240df9e395e43c8fa3dbd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.731980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0691ba899d0e404aa04465921d7e49e4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.741210 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0691ba899d0e404aa04465921d7e49e4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.754215 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1046/2082] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:45 2022] DELETE /identity/v3/users/1fc72a80660240df9e395e43c8fa3dbd => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:45.760779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.760779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.760779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.782386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:45.783866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:45.851941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1047/2083] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:45.861635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:45.863340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:45.868031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0691ba899d0e404aa04465921d7e49e4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.868857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0691ba899d0e404aa04465921d7e49e4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.893161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-207def50-a210-461f-81db-796c2a7ed5eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0691ba899d0e404aa04465921d7e49e4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:45.904446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:45.907872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1037/2084] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:45 2022] DELETE /identity/v3/users/0691ba899d0e404aa04465921d7e49e4 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:45.909239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:45.910923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:45.911712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:45.939125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:45.940683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:45.948139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88658714-e104-40b3-bca4-8444f64c1ce7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.027502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.028660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.028936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.029445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.031610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.047941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1038/2085] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:46.062054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.066333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.087479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f3121-c6d1-4576-b801-45b14fe9dadb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.096044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.096044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.096044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.096335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.107960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.109506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.123946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1039/2086] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:46.128171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.129762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.142678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1048/2087] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:46.789629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.793753 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.795758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:46.797666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:46.812048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e29759-7fdb-4c1c-b935-6749c475e7e3 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.820044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.820805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.821161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.821507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] PATH_INFO: `/v3/projects/4bf6b4394c2d4fc5bac3f426f3ee0b19` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.825927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf6b4394c2d4fc5bac3f426f3ee0b19)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.834065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.836858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6443d51-da2a-45a2-960e-80141ca6313d service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:46.848418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:46.850249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:46.850730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:46.851153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] PATH_INFO: `/v3/projects/38b7ecda83404572a25c0598c322f2e8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:46.853049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b7ecda83404572a25c0598c322f2e8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:46.855454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:46.994715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf6b4394c2d4fc5bac3f426f3ee0b19', 'request_id': 'req-5d66ca03-b295-4271-a20f-279f6124bcad'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.003786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:47.004887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf6b4394c2d4fc5bac3f426f3ee0b19', 'request_id': 'req-5d66ca03-b295-4271-a20f-279f6124bcad'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.026472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b7ecda83404572a25c0598c322f2e8', 'request_id': 'req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.102284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf6b4394c2d4fc5bac3f426f3ee0b19', 'request_id': 'req-5d66ca03-b295-4271-a20f-279f6124bcad'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.119035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] The token cache is being invalidate because project 4bf6b4394c2d4fc5bac3f426f3ee0b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:47.119784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.120203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.121943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1040/2088] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:46 2022] DELETE /identity/v3/projects/4bf6b4394c2d4fc5bac3f426f3ee0b19 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:47.126822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:47.130719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:47.184602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b7ecda83404572a25c0598c322f2e8', 'request_id': 'req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.197199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b7ecda83404572a25c0598c322f2e8', 'request_id': 'req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.222723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] The token cache is being invalidate because project 38b7ecda83404572a25c0598c322f2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:47.229876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.229876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.230130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66ca03-b295-4271-a20f-279f6124bcad admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:47.230241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1049/2089] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:46 2022] DELETE /identity/v3/projects/38b7ecda83404572a25c0598c322f2e8 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:47.231246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:47.233972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:47.258196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:47.326239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:47.328093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a2767-39b0-4ae7-afdf-9cc6f89f3b73 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:47.329754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:47.330046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:47.330326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] PATH_INFO: `/v3/users/da85961b9d564c889b8fa0c737e97115` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:47.331742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da85961b9d564c889b8fa0c737e97115)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:47.333647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:47.347914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:47.347914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:47.347914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:47.347914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] PATH_INFO: `/v3/users/f3dab808e10a4acbabf5225c986c5ac8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:47.348996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dab808e10a4acbabf5225c986c5ac8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:47.351298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:47.552030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dab808e10a4acbabf5225c986c5ac8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.557202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da85961b9d564c889b8fa0c737e97115'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.568107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dab808e10a4acbabf5225c986c5ac8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.579781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da85961b9d564c889b8fa0c737e97115'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.579966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dab808e10a4acbabf5225c986c5ac8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.581228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dab808e10a4acbabf5225c986c5ac8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.582053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f208ad24-b4b4-456e-9b90-be54deb643ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dab808e10a4acbabf5225c986c5ac8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.662219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da85961b9d564c889b8fa0c737e97115'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.663024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da85961b9d564c889b8fa0c737e97115'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.676175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da85961b9d564c889b8fa0c737e97115'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:47.682265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1041/2090] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:47 2022] DELETE /identity/v3/users/da85961b9d564c889b8fa0c737e97115 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:47.726178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1050/2091] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:47 2022] DELETE /identity/v3/users/f3dab808e10a4acbabf5225c986c5ac8 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:48.144072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:48.146701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.175959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:48.229702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9908-9df2-49b6-8e68-75fcbba237f4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:48.351433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:48.351986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.352256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.352530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] PATH_INFO: `/v3/users/bb6e38cb0019468ba7bb0962af5b5445` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.354495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6e38cb0019468ba7bb0962af5b5445)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:48.356548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:48.398556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.401917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.402293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.425883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] MFA Rules not processed for user `881062ec3efc4445a43e1ff772b09153`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:48.428209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.510630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1051/2092] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:48.511509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:48.512849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.536184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d860264-8a06-4d89-ac15-992f38687094 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:48.541405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:48.542074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.542777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.542777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] PATH_INFO: `/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.544256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf94fc270821436dbc34b7ee3a0af3a2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:48.546538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:48.601193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6e38cb0019468ba7bb0962af5b5445'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.610439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6e38cb0019468ba7bb0962af5b5445'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.644327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:48.663763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf94fc270821436dbc34b7ee3a0af3a2', 'request_id': 'req-8d35a7d0-016f-478c-b88f-79179f9e8147'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.691452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6e38cb0019468ba7bb0962af5b5445'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.692362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6e38cb0019468ba7bb0962af5b5445'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.709222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6e38cb0019468ba7bb0962af5b5445'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.716541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1042/2093] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:48 2022] DELETE /identity/v3/users/bb6e38cb0019468ba7bb0962af5b5445 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:48.717396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:48.719020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.773735 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:48.787644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf94fc270821436dbc34b7ee3a0af3a2', 'request_id': 'req-8d35a7d0-016f-478c-b88f-79179f9e8147'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.798637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6c6c5-3cc5-43aa-96c0-bfa5d0114ce0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:48.799763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf94fc270821436dbc34b7ee3a0af3a2', 'request_id': 'req-8d35a7d0-016f-478c-b88f-79179f9e8147'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.805425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:48.806092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.806363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.806654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] PATH_INFO: `/v3/projects/78fd50775c7f4bd0a058e102ef23c26b` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.807794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fd50775c7f4bd0a058e102ef23c26b)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:48.809715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:48.818254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] The token cache is being invalidate because project bf94fc270821436dbc34b7ee3a0af3a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:48.819082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.819926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8d35a7d0-016f-478c-b88f-79179f9e8147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.821608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1052/2094] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:48 2022] DELETE /identity/v3/projects/bf94fc270821436dbc34b7ee3a0af3a2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:48.825878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d45b0-19cf-4a42-badb-34659e1a0309 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.826194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d45b0-19cf-4a42-badb-34659e1a0309 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.829335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d45b0-19cf-4a42-badb-34659e1a0309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.851824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-424d45b0-19cf-4a42-badb-34659e1a0309 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:48.852971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-424d45b0-19cf-4a42-badb-34659e1a0309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.861620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1053/2095] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:48.863025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:48.863380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:48.863729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:48.883690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:48.886575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.897861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1054/2096] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:48.899071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:48.905697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:48.940860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fd50775c7f4bd0a058e102ef23c26b', 'request_id': 'req-3675040e-e95f-462f-b4c7-c6c37a28c464'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.949570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fd50775c7f4bd0a058e102ef23c26b', 'request_id': 'req-3675040e-e95f-462f-b4c7-c6c37a28c464'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:48.974725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22664215-f2e1-48b2-aec7-a069f787d48e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.016128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fd50775c7f4bd0a058e102ef23c26b', 'request_id': 'req-3675040e-e95f-462f-b4c7-c6c37a28c464'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.033156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.033804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.034053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.034321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.057973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] The token cache is being invalidate because project 78fd50775c7f4bd0a058e102ef23c26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:49.059221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.060763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3675040e-e95f-462f-b4c7-c6c37a28c464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:49.065168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1043/2097] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:48 2022] DELETE /identity/v3/projects/78fd50775c7f4bd0a058e102ef23c26b => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:49.068292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad8a-e8c3-45b7-b216-24882425fba1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.068788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad8a-e8c3-45b7-b216-24882425fba1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.069489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28ad8a-e8c3-45b7-b216-24882425fba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.104398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-eb28ad8a-e8c3-45b7-b216-24882425fba1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:49.105933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-eb28ad8a-e8c3-45b7-b216-24882425fba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.206444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.210501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c2ea1-1281-4512-a9c2-24bef055dfc0 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.227451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1055/2098] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:49.228802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.229092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.229988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.261112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:49.263999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.273347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1056/2099] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:49.279368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1044/2100] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:49.280862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.281312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.281610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.281905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.292862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:49.299876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c5aef-817b-4467-a1fa-1777085daf36 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.313316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.313944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.314244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.314542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.315163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:49.315274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:49.315467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.316388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.317114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.326505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1045/2101] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:49.332017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.333854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1057/2102] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:49.338625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.345828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720ba01-3920-43e0-a88c-d28f032b9942 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.358140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.358140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.358140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.358520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c7ea6-dcea-4dad-8138-4588ccd3bb5d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.358611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.358611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:49.358611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.358611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.362115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.362896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.363264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.363531 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.364535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.369701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1046/2103] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:49.369830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.374661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.393316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0957a-f088-42d3-99f1-2fe94c3b7c86 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.409142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.415774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.415774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.415774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.415774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.415774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.532111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1058/2104] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/projects => generated 371 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:49.533586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.550536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1047/2105] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:49.554120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.626200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:49.671855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0a3f0-bff4-4fc3-ac54-fd1b6d68c14a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.687127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.687857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.695102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.695102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.695102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.695373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d8ae5-3956-40b3-a579-b4361e6d9ab4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.695460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.706687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.707360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.707652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.707925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.708640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.711696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.808078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1059/2106] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/users => generated 286 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:49.808078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.817665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21e939-39b3-4899-82fb-d7f2dd972887 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.822801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.823628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.823906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.824176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.824812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.826665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.838260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1060/2107] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:49.844671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.854797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055f8b3-09ac-4570-86e6-a7341d325977 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.860944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.861533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.861817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.862094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.862815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.864911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.882460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1048/2108] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:49 2022] POST /identity/v3/users => generated 289 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:49.882787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1061/2109] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:49.885247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.888705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.903688 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6c13-382f-417d-bbba-7e768b68d4d9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.907538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b402a-4991-4ebf-8071-06d7a193c91d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.909561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.910336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.910917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.911262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.911901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.913377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.913578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.913937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.914224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.914524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] PATH_INFO: `/v3/projects/f396dc6bd79f495880bb8db13482b6e8/users/8b07de2df1884b7186e9656c6a95bb09/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.927940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1049/2110] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:49.935793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:49.935793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:49.936408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f396dc6bd79f495880bb8db13482b6e8, user_id=8b07de2df1884b7186e9656c6a95bb09, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.937120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:49.946058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3784-36b7-41b4-bcb9-544642c51a0f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:49.951643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:49.951997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:49.952275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:49.952543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] PATH_INFO: `/v3/projects/cad65f0671c74411a13bc1d12229e8f9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:49.953661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad65f0671c74411a13bc1d12229e8f9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:49.955879 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.019461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1062/2111] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:49 2022] PUT /identity/v3/projects/f396dc6bd79f495880bb8db13482b6e8/users/8b07de2df1884b7186e9656c6a95bb09/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:50.019461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.083322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef415-54c8-4b76-bed9-9393f4f49303 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.085948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad65f0671c74411a13bc1d12229e8f9', 'request_id': 'req-4c6a346c-1f68-49fe-94f3-082055d71e93'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.092468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad65f0671c74411a13bc1d12229e8f9', 'request_id': 'req-4c6a346c-1f68-49fe-94f3-082055d71e93'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.100718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:50.139873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.140399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.140669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.140937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.141593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.143205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd2d6a-b95b-4f55-b9d5-75742eb16426 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.157812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1063/2112] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:50.157812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a950af-fb08-4d52-bcf3-f854e6f4bf4b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.157812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a950af-fb08-4d52-bcf3-f854e6f4bf4b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.157812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a950af-fb08-4d52-bcf3-f854e6f4bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.165203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad65f0671c74411a13bc1d12229e8f9', 'request_id': 'req-4c6a346c-1f68-49fe-94f3-082055d71e93'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.187676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] The token cache is being invalidate because project cad65f0671c74411a13bc1d12229e8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:50.187676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.187676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:50.187676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1050/2113] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:49 2022] DELETE /identity/v3/projects/cad65f0671c74411a13bc1d12229e8f9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:50.187676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.189391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.196278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-49a950af-fb08-4d52-bcf3-f854e6f4bf4b None None] MFA Rules not processed for user `8b07de2df1884b7186e9656c6a95bb09`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.198230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-49a950af-fb08-4d52-bcf3-f854e6f4bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.204395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a346c-1f68-49fe-94f3-082055d71e93 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.209247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.209936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.210217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.210524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] PATH_INFO: `/v3/projects/065aa1d575f94b3e98f7d9e1e1e47ce9/users/e119e37ecbc44b628e79b2c9459b2f8a/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.227782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065aa1d575f94b3e98f7d9e1e1e47ce9, user_id=e119e37ecbc44b628e79b2c9459b2f8a, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.230509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b00dc-c4d8-41f2-9718-5deecfcf46a4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.271226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1064/2114] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:50.272506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8e3c-6ed4-463f-9bdc-17267f6e68be None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.273141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8e3c-6ed4-463f-9bdc-17267f6e68be None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.278248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec8e3c-6ed4-463f-9bdc-17267f6e68be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.296188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ccec8e3c-6ed4-463f-9bdc-17267f6e68be None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.297448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ccec8e3c-6ed4-463f-9bdc-17267f6e68be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.305612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1065/2115] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:50.314860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.315306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.315588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.335568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.336759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.340928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1051/2116] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:50 2022] PUT /identity/v3/projects/065aa1d575f94b3e98f7d9e1e1e47ce9/users/e119e37ecbc44b628e79b2c9459b2f8a/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:50.351216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.351496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.351768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.382139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] MFA Rules not processed for user `e119e37ecbc44b628e79b2c9459b2f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:50.383393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.401108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1066/2117] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:50.405403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.414789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d5c50-82db-4530-b12c-35dcd1a9d946 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.420469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.421242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.421670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.422114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.422938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:50.423246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.424920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.438621 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1067/2118] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:50.443468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.451695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:50.455151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8b203-0b33-4936-84a5-1cebfe443bd4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.460837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.461437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.461711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.461980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.462071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1052/2119] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:50.462636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.464783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.612950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.613510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1068/2120] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:50.625833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.628294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.631125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143684b-8e58-4814-b6ac-c6b5c91346ab None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.646771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:50.716501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.717191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.717493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a509757-ebab-4e3e-bee8-105e4e76e5ba admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.717600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.717904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.718676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.725146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.725468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.725468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.725468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.725468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.784979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.786429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.796822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1069/2121] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:50.797515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.799267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.825743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef978-5305-4a65-a715-533a9d23bb37 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.831498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1053/2122] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:50 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:50.832634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.834804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:50.839939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.840582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.840857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.841170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.852809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca52588-1758-468b-99ca-8e5cd17e82ba admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.859989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.860701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.862011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.863321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] PATH_INFO: `/v3/users/8b07de2df1884b7186e9656c6a95bb09` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.864102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b07de2df1884b7186e9656c6a95bb09)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.866573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.928881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.930648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.941885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1070/2123] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:30:50.943511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.954261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d9752-b685-433b-b829-d71bc28eef1b service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.959930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.960096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.960234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.960516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.961144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:50.963172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:50.978450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1071/2124] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:50.979932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:50.992128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610bca4-bffa-44e0-bb03-cfae26791a5e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:50.997235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:50.997946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:50.998275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:50.998595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:50.999240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.000832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.012012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1072/2125] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:51.016632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.026920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e7d2-dfcc-47b0-8a4c-f69e92a185fd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.033159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.033809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.034092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.045855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] PATH_INFO: `/v3/projects/ac8b09442973422a8fc6d5e210ffd583/users/8a311a9ff74a44e3b944ce33854cb59e/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.049936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8b09442973422a8fc6d5e210ffd583, user_id=8a311a9ff74a44e3b944ce33854cb59e, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.052842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2170-456a-485c-9bcf-3f72a0271664 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.160358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1073/2126] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:51 2022] PUT /identity/v3/projects/ac8b09442973422a8fc6d5e210ffd583/users/8a311a9ff74a44e3b944ce33854cb59e/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:51.161951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.162238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.162544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.167265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b07de2df1884b7186e9656c6a95bb09'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:51.175085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b07de2df1884b7186e9656c6a95bb09'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:51.183024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:51.190904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] MFA Rules not processed for user `8a311a9ff74a44e3b944ce33854cb59e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:51.192120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.256154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1074/2127] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:51.260687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.275016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b07de2df1884b7186e9656c6a95bb09'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:51.275655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b07de2df1884b7186e9656c6a95bb09'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:51.301718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d153be25-2c82-4d60-94fc-a12a9e1a58cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b07de2df1884b7186e9656c6a95bb09'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:51.309782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1054/2128] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:30:50 2022] DELETE /identity/v3/users/8b07de2df1884b7186e9656c6a95bb09 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:51.314696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.314980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.315254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.332554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a2db9-4667-4293-af47-406e7080dc03 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.335398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:51.337265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.384258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:51.389799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.390538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.390822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.391097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.391730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.393679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.396371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1055/2129] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:51.406456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.407842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.468654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ab7a8-e2de-4041-8a9d-d6cb7ebe2c49 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.472255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.472854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.473145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.473399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.482695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.484162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.493831 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1056/2130] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:51.513182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1075/2131] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/projects => generated 393 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:51.514968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.516258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:51.537403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca576a27-eb3a-4faa-8f38-7f9d91e57b01 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.600452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.601073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.601346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.602390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.603061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.605019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.766882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1057/2132] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:51 2022] POST /identity/v3/users => generated 303 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:51.770674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.785393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa7cc2-8ede-4feb-afb1-067743a93ed8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.799086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.799718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.800060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.800337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.801012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.803278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:51.816753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1076/2133] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:51.821062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:51.846056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe15361-ff1c-4f5f-a846-044866eaf55b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:51.853948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:51.854660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:51.854938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:51.859690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] PATH_INFO: `/v3/projects/94f92a1931354df5aca2465f91881598/users/002dc9dafca64af3b2e45eee2a8e0d46/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:51.871644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f92a1931354df5aca2465f91881598, user_id=002dc9dafca64af3b2e45eee2a8e0d46, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:51.874389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.004307 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1058/2134] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:51 2022] PUT /identity/v3/projects/94f92a1931354df5aca2465f91881598/users/002dc9dafca64af3b2e45eee2a8e0d46/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:52.009299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.019578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e4863-aead-48d4-b72c-c8e03fb26221 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.038952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:52.076618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.078713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.080127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.081241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.081241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.081444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.081955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.083629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.092458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293ba1d-3edf-42c9-badf-e6dddc123487 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.094795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1077/2135] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:52.099759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.115702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.116428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.116747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.117100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] PATH_INFO: `/v3/projects/f396dc6bd79f495880bb8db13482b6e8` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.117552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f10ce-b097-4f29-93b9-14b30ebb9cce admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.119768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f396dc6bd79f495880bb8db13482b6e8)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.121700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.123940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.124694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.125162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.125444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] PATH_INFO: `/v3/projects/94f92a1931354df5aca2465f91881598/users/002dc9dafca64af3b2e45eee2a8e0d46/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.127646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f92a1931354df5aca2465f91881598, user_id=002dc9dafca64af3b2e45eee2a8e0d46, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.130798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e799dd0-064f-48b8-ab21-12ea7859fb93 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.143642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1078/2136] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:52 2022] PUT /identity/v3/projects/94f92a1931354df5aca2465f91881598/users/002dc9dafca64af3b2e45eee2a8e0d46/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:52.154878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.155319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.155599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.188509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] MFA Rules not processed for user `002dc9dafca64af3b2e45eee2a8e0d46`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:52.189753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.267752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1079/2137] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:30:52.281042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.281544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f396dc6bd79f495880bb8db13482b6e8', 'request_id': 'req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.282858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.288452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f396dc6bd79f495880bb8db13482b6e8', 'request_id': 'req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.290241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:52.346636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbada2b-cae1-4314-8ac7-88d905a05799 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.399179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f396dc6bd79f495880bb8db13482b6e8', 'request_id': 'req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.415811 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] The token cache is being invalidate because project f396dc6bd79f495880bb8db13482b6e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:30:52.416111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.418383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-756b2769-cf2a-4d4a-b2cd-f345ea8e238c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:30:52.420347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:52.424786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1059/2138] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:30:52 2022] DELETE /identity/v3/projects/f396dc6bd79f495880bb8db13482b6e8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:30:52.426445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2240a-1350-43d4-9fce-73066b512181 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.426800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2240a-1350-43d4-9fce-73066b512181 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.431617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2240a-1350-43d4-9fce-73066b512181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.434661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.435899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.436255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.437706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.441276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.449972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a8d2240a-1350-43d4-9fce-73066b512181 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:52.451961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a8d2240a-1350-43d4-9fce-73066b512181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.537907 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1060/2139] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:52.545564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.546054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.546563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.561242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.562850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.573434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1080/2140] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:52.615647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:52.620554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.628862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1061/2141] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:52.632734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.648606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.650239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.653282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1a380-dd1c-4b2e-87e5-2a1adf5f089d service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.665105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b00a7-2987-4694-a0b4-7ab377610361 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.667227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.667866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.668152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.668425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.669550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:30:52.669705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.670071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.670433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.670785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.671134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.673516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:52.674662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.688442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1081/2142] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:30:52.692374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.707471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932ab6-5484-40ec-b03d-2082a0a67ab8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:52.712781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:52.713392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:52.713671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:52.713946 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:52.714611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.716908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.768520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:52.770745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:52.783939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1062/2143] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:30:52.855811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1082/2144] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:30:52.856567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:52.875835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69a5c6-ef0a-48b8-b445-5c390b461adb service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.000557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.001432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.001716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.001989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.002701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.005345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.116703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1063/2145] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:30:52 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:30:53.117625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.129064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347df99-472a-4745-b6c5-1a30985c79f8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.134294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.135204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.135661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.138647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.138647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.139053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.151070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1083/2146] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:30:53.155336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.167442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087ae7b-dff9-42b2-9578-644044ddbe73 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.173058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.173668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.173947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.174218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.175022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.176646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a8dc8-c182-41ff-81d7-06f1326b87c1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.178915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:30:53.191851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1064/2147] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:30:53.193092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.205194 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3f848-ecd9-436d-8ad5-8c5862610ba9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.212232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.212892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.213171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.213446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] PATH_INFO: `/v3/projects/0edae1f8e98b411dbd104d9e33ae1a95/users/7a38fef6e7fd4475a31403d2333b8e64/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.228824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edae1f8e98b411dbd104d9e33ae1a95, user_id=7a38fef6e7fd4475a31403d2333b8e64, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.231629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.331932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1084/2148] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:30:53 2022] PUT /identity/v3/projects/0edae1f8e98b411dbd104d9e33ae1a95/users/7a38fef6e7fd4475a31403d2333b8e64/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:30:53.344593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.345174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.345770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.380619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] MFA Rules not processed for user `7a38fef6e7fd4475a31403d2333b8e64`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:30:53.381967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.449808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1065/2149] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:30:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:30:53.484437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:30:53.486236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:30:53.564966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec34a88-6117-4247-9647-334095dca8c5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:30:53.568767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:30:53.569377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:30:53.569654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:30:53.569922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:30:53.579495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:30:53.580923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53685838-d11a-4ba9-81d1-c501ef4e9014 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:30:53.591361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1085/2150] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:30:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:03.586294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:04.632890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=33 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:08.454447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:08.456039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:08.522467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd2dc1-c2a7-4c25-85bf-53bf9efb3123 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:08.527672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:08.528485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.528869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.529267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] PATH_INFO: `/v3/users/8a311a9ff74a44e3b944ce33854cb59e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.530519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a311a9ff74a44e3b944ce33854cb59e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:08.533389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:08.831021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a311a9ff74a44e3b944ce33854cb59e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:08.838193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a311a9ff74a44e3b944ce33854cb59e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:08.923462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a311a9ff74a44e3b944ce33854cb59e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:08.924208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a311a9ff74a44e3b944ce33854cb59e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:08.937242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a311a9ff74a44e3b944ce33854cb59e'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:08.943340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1066/2151] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:08 2022] DELETE /identity/v3/users/8a311a9ff74a44e3b944ce33854cb59e => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:08.947888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:08.948352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:08.948482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:08.968663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:08.970526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:09.028513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1086/2152] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:09.038332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:09.039785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:09.118077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76e361-b962-4874-9656-31fd01dbedc9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:09.121851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:09.122529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:09.122743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:09.123019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:09.134157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:09.135717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:09.146991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1067/2153] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:09.718502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:09.722586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:09.734163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f0782-e8da-4095-96d1-de52261a6768 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:09.739943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:09.740586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:09.740868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:09.741190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] PATH_INFO: `/v3/projects/ac8b09442973422a8fc6d5e210ffd583` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:09.742354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8b09442973422a8fc6d5e210ffd583)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:09.745464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:09.887725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8b09442973422a8fc6d5e210ffd583', 'request_id': 'req-9d249a54-b1ab-41bb-9317-a928fba08796'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:09.946230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8b09442973422a8fc6d5e210ffd583', 'request_id': 'req-9d249a54-b1ab-41bb-9317-a928fba08796'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:09.952277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8b09442973422a8fc6d5e210ffd583', 'request_id': 'req-9d249a54-b1ab-41bb-9317-a928fba08796'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:09.963935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] The token cache is being invalidate because project ac8b09442973422a8fc6d5e210ffd583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:09.964167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:09.964799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:09.966290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1087/2154] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:09 2022] DELETE /identity/v3/projects/ac8b09442973422a8fc6d5e210ffd583 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:09.970780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:09.981269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502eed3-d506-4767-b3b7-5a1358290f77 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:10.003533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:10.043255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:10.044253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:10.044538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:10.044868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] PATH_INFO: `/v3/users/002dc9dafca64af3b2e45eee2a8e0d46` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:10.046444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002dc9dafca64af3b2e45eee2a8e0d46)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:10.049130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:10.188441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002dc9dafca64af3b2e45eee2a8e0d46'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.195625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002dc9dafca64af3b2e45eee2a8e0d46'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.272246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002dc9dafca64af3b2e45eee2a8e0d46'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.276522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002dc9dafca64af3b2e45eee2a8e0d46'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.286928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002dc9dafca64af3b2e45eee2a8e0d46'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.292954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1068/2155] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:09 2022] DELETE /identity/v3/users/002dc9dafca64af3b2e45eee2a8e0d46 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:10.585574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:10.587111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:10.600596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d249a54-b1ab-41bb-9317-a928fba08796 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:10.683908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:10.684548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:10.684823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:10.685090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] PATH_INFO: `/v3/users/7ed9ce5ab1c34b9797aea3d851baf8a4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:10.686240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed9ce5ab1c34b9797aea3d851baf8a4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:10.688867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:10.782985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:10.786533 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:10.810840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:10.826295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a017f8b-0623-4ea6-8069-3ae363c901e2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:10.840738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:10.841713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:10.841979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:10.842261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] PATH_INFO: `/v3/projects/94f92a1931354df5aca2465f91881598` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:10.843488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f92a1931354df5aca2465f91881598)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:10.847467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:10.925472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed9ce5ab1c34b9797aea3d851baf8a4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.937242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed9ce5ab1c34b9797aea3d851baf8a4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.944868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed9ce5ab1c34b9797aea3d851baf8a4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.945966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed9ce5ab1c34b9797aea3d851baf8a4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:10.946603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8a8ac387-e4e3-4769-af66-2988ec0f13fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed9ce5ab1c34b9797aea3d851baf8a4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.003871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f92a1931354df5aca2465f91881598', 'request_id': 'req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.009376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f92a1931354df5aca2465f91881598', 'request_id': 'req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.050361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1088/2156] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:10 2022] DELETE /identity/v3/users/7ed9ce5ab1c34b9797aea3d851baf8a4 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:11.054606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.054886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.055168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.074312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:11.075553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.134840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f92a1931354df5aca2465f91881598', 'request_id': 'req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.135130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1089/2157] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:11.144967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.146584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.146919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] The token cache is being invalidate because project 94f92a1931354df5aca2465f91881598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:11.147206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.147846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-73d4fbc6-666d-4c71-a020-b5566dcb4b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.149498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1069/2158] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:10 2022] DELETE /identity/v3/projects/94f92a1931354df5aca2465f91881598 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:11.153963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77425a6f-e081-4881-81a7-14cc35a1a7b8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.154249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77425a6f-e081-4881-81a7-14cc35a1a7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.154560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77425a6f-e081-4881-81a7-14cc35a1a7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.174705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-77425a6f-e081-4881-81a7-14cc35a1a7b8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:11.175898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-77425a6f-e081-4881-81a7-14cc35a1a7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.183740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1070/2159] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:11.191878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.192159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.192428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.204636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc85b8-78ea-442e-b3ba-2ce7d281a9d5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.208790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.209084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.209488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.209769 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.211766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.211981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:11.213228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.222341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1071/2160] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:11.224432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.225868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.226667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.237328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1090/2161] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:11.238699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8bf3-075b-493b-82f5-326a78e075fb None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.243649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.244557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.244557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.244764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.245425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:11.245716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.247435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.256137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:11.267072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1072/2162] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:11.268705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.281073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ffd9-4e69-44af-ade5-623e9495285b service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.286553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.287157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.287437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.287713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.288341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.290255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.467520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1091/2163] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/projects => generated 383 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:11.468786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.479853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864b9a1-ca97-4f82-a644-c4eb88adda6e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.587987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.588186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.588828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.589331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.589698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.589811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.591042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.593496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.623421 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb54dc3-92f4-4d39-a4c5-22066a8619e8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.643519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.643519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.643519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.643519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] PATH_INFO: `/v3/users/7a38fef6e7fd4475a31403d2333b8e64` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.644394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a38fef6e7fd4475a31403d2333b8e64)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.649141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.741124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:11.747884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1073/2164] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:11 2022] POST /identity/v3/users => generated 292 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:11.752521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:11.752521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:11.763226 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae7f2d-e446-4382-aa6a-b90c4ee4df8e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:11.768756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:11.769396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:11.769707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:11.769993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] PATH_INFO: `/v3/projects/690ee9d980b44445941cb08d4f84904c` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:11.771230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] RBAC: Authorizing `identity:delete_project(project_id=690ee9d980b44445941cb08d4f84904c)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:11.773165 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:11.939543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a38fef6e7fd4475a31403d2333b8e64'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.941852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690ee9d980b44445941cb08d4f84904c', 'request_id': 'req-ce04a627-2f74-4858-8fdb-99dcebdf426b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.952224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690ee9d980b44445941cb08d4f84904c', 'request_id': 'req-ce04a627-2f74-4858-8fdb-99dcebdf426b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.962736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a38fef6e7fd4475a31403d2333b8e64'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.975193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a38fef6e7fd4475a31403d2333b8e64'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.976219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a38fef6e7fd4475a31403d2333b8e64'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:11.976937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de4984be-2d62-40f7-a100-a4be0f329d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a38fef6e7fd4475a31403d2333b8e64'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:12.037894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690ee9d980b44445941cb08d4f84904c', 'request_id': 'req-ce04a627-2f74-4858-8fdb-99dcebdf426b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:12.052139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] The token cache is being invalidate because project 690ee9d980b44445941cb08d4f84904c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:12.052426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:12.053147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:12.054656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1074/2165] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:11 2022] DELETE /identity/v3/projects/690ee9d980b44445941cb08d4f84904c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:12.055432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.056754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.106785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1092/2166] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:11 2022] DELETE /identity/v3/users/7a38fef6e7fd4475a31403d2333b8e64 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:12.111963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682804-7237-40e5-a24d-3bade0a00ed8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.112252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682804-7237-40e5-a24d-3bade0a00ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.112529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1682804-7237-40e5-a24d-3bade0a00ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.131376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04a627-2f74-4858-8fdb-99dcebdf426b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.133052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c1682804-7237-40e5-a24d-3bade0a00ed8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:12.134528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c1682804-7237-40e5-a24d-3bade0a00ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.144822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:12.190716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.191461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.191742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.192015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.192662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.194336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57668c76-7bf4-45b7-ba10-f3c8e88d60b8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.196835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1093/2167] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:12.198079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.198361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.198663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.206970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1075/2168] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:12.208333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.208619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.208890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.219905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:12.221113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.231672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1094/2169] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:12.231672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.232725 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:12.233974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.242671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1076/2170] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:12.246647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e7044-2d6e-4a9d-bb6e-1c0134cda23b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.251035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.251035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.252493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.253184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.253460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.253724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.254489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.256509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.267260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1095/2171] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:12.268415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.277879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f6b47-ba00-479f-afba-313071dfdb18 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.282762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.283362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.283739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.283934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.284568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:12.284928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.286741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.301134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1096/2172] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:12.302328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.313403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394fc9c-d264-4d73-b424-28287f1090f5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.317377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390fc8e-c940-4a9b-95ed-6c2b5bea6e22 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.318693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.319530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.320606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.320606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] PATH_INFO: `/v3/projects/6a17d26e83d64847b5fa052862e6dde8/users/ca54a9af332e48bbb53f20569cae49dd/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.322912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.328041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.328041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.328041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.336002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a17d26e83d64847b5fa052862e6dde8, user_id=ca54a9af332e48bbb53f20569cae49dd, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.337362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.338782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f118b81-efdc-45ad-9b91-7e44f1072907 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.339009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.349109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1077/2173] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:12.351459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.362847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5c5d8-5b02-4a16-910d-b8fc01350b93 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.368322 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.368951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.369220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.369493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.370134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.372862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.427278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1097/2174] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:12 2022] PUT /identity/v3/projects/6a17d26e83d64847b5fa052862e6dde8/users/ca54a9af332e48bbb53f20569cae49dd/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:12.440873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.441515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.441794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.470246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] MFA Rules not processed for user `ca54a9af332e48bbb53f20569cae49dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:12.471596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.487044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1078/2175] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/projects => generated 391 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:12.497894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.508284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:12.551518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1098/2176] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:12.555513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.572454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d46db7-fff6-4ee7-a2c0-caefdf7e57af admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.578475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.579121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.579396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.579666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.580302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.582540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.642490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc7b01-ec0b-4123-b835-34d53c340679 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.647280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.647954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.648230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.648490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.649125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.651057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.690130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1079/2177] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:12.694063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.703872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca252c-fe0d-4346-a82b-6a31647f31d8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.708545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.709113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.709389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.709661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.710353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.711980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.722363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1080/2178] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:12.726084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.736030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887ebc0-e610-4097-908f-694341fc1f07 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.740579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.741140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.741436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.741699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.742317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.743898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.745162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:12.756566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1081/2179] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:12.765044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.767998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1099/2180] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:12.771725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.782460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee51b65-0f67-4078-a4b5-37b5426d5718 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.856487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a921-e01b-492f-bc67-b8a2742cec87 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.863105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.864333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.864776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:12.865775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.865775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.865955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:12.865955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:12.866442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] PATH_INFO: `/v3/projects/fec5165397344d70aecfdc82ca872557/users/02dc36aabf204d6795f5b32254fef5ea/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:12.866578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.868933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.885001 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec5165397344d70aecfdc82ca872557, user_id=02dc36aabf204d6795f5b32254fef5ea, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:12.888040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:12.969587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1100/2181] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:12 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:12.970596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:12.972251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:12.985285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2f1f-4cf1-42c9-91d0-5c2d0dde39a4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:12.992444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1082/2182] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:12 2022] PUT /identity/v3/projects/fec5165397344d70aecfdc82ca872557/users/02dc36aabf204d6795f5b32254fef5ea/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:12.993359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.032238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:13.047898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.048535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.048814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.049091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] PATH_INFO: `/v3/projects/0edae1f8e98b411dbd104d9e33ae1a95` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.050299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edae1f8e98b411dbd104d9e33ae1a95)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.052259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.067917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89f9c4-945d-4489-999a-0172a62fb590 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.069491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.070194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.070571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.070911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.071622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.073302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae69a74-07c1-4ca4-a106-7162c440db89 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.083503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1083/2183] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:13.084752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.085038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.085306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.113517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] MFA Rules not processed for user `02dc36aabf204d6795f5b32254fef5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:13.114759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.170239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edae1f8e98b411dbd104d9e33ae1a95', 'request_id': 'req-338be72a-a044-48d6-86c5-ce22fe5a49c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:13.184351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1084/2184] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:13.185198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.197563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:13.202032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb6379-d7e6-4561-9443-5e1aa3f4647c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.259957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.260613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.260886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.261180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] PATH_INFO: `/v3/projects/f3cded48b4204bebb4f43eb62b53ce50/users/90f7c284a192423a960f80944c9b0a45/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.276116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cded48b4204bebb4f43eb62b53ce50, user_id=90f7c284a192423a960f80944c9b0a45, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.278822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99795d52-a220-4019-9df1-e1113f2fc874 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.317428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edae1f8e98b411dbd104d9e33ae1a95', 'request_id': 'req-338be72a-a044-48d6-86c5-ce22fe5a49c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:13.323634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edae1f8e98b411dbd104d9e33ae1a95', 'request_id': 'req-338be72a-a044-48d6-86c5-ce22fe5a49c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:13.336809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] The token cache is being invalidate because project 0edae1f8e98b411dbd104d9e33ae1a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:13.337101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:13.337768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:13.339380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1101/2185] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:12 2022] DELETE /identity/v3/projects/0edae1f8e98b411dbd104d9e33ae1a95 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:13.352969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.352969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.392343 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1085/2186] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:13 2022] PUT /identity/v3/projects/f3cded48b4204bebb4f43eb62b53ce50/users/90f7c284a192423a960f80944c9b0a45/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:13.395378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.395378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.395378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.405195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338be72a-a044-48d6-86c5-ce22fe5a49c2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.413984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:13.416019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.462993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.463855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.464294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.464747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.466816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.479894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1086/2187] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:13.487990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.487990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.495398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d96d4-b1c1-4dbb-93ed-9bdbde342a75 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.500936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.501909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.502192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.502488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.503254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.504864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5d74-61d5-4c3e-b9ab-e6a9a753d931 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.519348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1087/2188] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:13.519348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.519348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.519348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.525951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:13.540797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.541832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:13.542237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.543219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.551885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1102/2189] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:13.552881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.557960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1088/2190] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:13.559609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.564631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5473d5-77d1-4b34-9fcc-dfd1649c411d service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.570371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ece96-f0c3-415c-bfed-c768d0dde670 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.572349 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.574102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.574581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.576167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] PATH_INFO: `/v3/projects/f3cded48b4204bebb4f43eb62b53ce50/users/90f7c284a192423a960f80944c9b0a45/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.576546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.577158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.578296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.578296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.579357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:13.579685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.581210 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cded48b4204bebb4f43eb62b53ce50, user_id=90f7c284a192423a960f80944c9b0a45, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.581329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.584292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731a7d5-36d4-470a-930c-5b4666dc02f6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.595403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1089/2191] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:13.599392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.602294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1103/2192] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:13 2022] PUT /identity/v3/projects/f3cded48b4204bebb4f43eb62b53ce50/users/90f7c284a192423a960f80944c9b0a45/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:13.611935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d7bf9-dcdb-4c94-9a9c-703ea9080c65 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.614050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797b09-174d-47f6-b020-af618522ffca None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.614295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797b09-174d-47f6-b020-af618522ffca None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.614611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797b09-174d-47f6-b020-af618522ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.659939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-13797b09-174d-47f6-b020-af618522ffca None None] MFA Rules not processed for user `90f7c284a192423a960f80944c9b0a45`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:13.661286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-13797b09-174d-47f6-b020-af618522ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.674368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:13.680534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.681143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.682878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.682878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.682878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.684388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.735554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1104/2193] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:13.746100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13797b09-174d-47f6-b020-af618522ffca None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.747505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-13797b09-174d-47f6-b020-af618522ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:13.803965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13797b09-174d-47f6-b020-af618522ffca None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.807639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.808245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.808525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.808796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.816351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1090/2194] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:13.818722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.820276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.820400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.831680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1105/2195] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:13.887400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b485181-e913-409d-9b54-18360b3303ec admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.892327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.894283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.894283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:13.894283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:13.894283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:13.896073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:13.936292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.945743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b54f78-cdd0-4f4f-b531-5f31f2b376a7 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.983826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1091/2196] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:13 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:13.988102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:13.998585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aad143-d955-4fc6-8d64-08a9fc10047d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:13.998837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:13.999452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:13.999810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.000021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.002600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:14.005724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.007469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.007469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.007469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.007890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.009727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.018117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:14.025281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1092/2197] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:14.026378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:14.038298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0f31-91d1-469f-ac4d-4b6a7e0413b2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:14.044329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.044943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.045215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.045544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.046192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.048468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.062349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1093/2198] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:14.065203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:14.074594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd671863-8ce7-4944-a688-15c9e11eb02e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:14.078008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.079785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.079959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558f094d-8722-4e70-bfa7-a5373456e725 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.080573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.080921 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.081263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] PATH_INFO: `/v3/projects/848fcac123a34c39b9111d865e577e3c/users/176dc8fd7de44108a331ac50b57eb852/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.093785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1106/2199] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:14.097266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848fcac123a34c39b9111d865e577e3c, user_id=176dc8fd7de44108a331ac50b57eb852, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.100250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.176311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1094/2200] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:14 2022] PUT /identity/v3/projects/848fcac123a34c39b9111d865e577e3c/users/176dc8fd7de44108a331ac50b57eb852/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:14.186776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.187593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.188427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.257199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] MFA Rules not processed for user `176dc8fd7de44108a331ac50b57eb852`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:14.258481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:14.336912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1107/2201] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:14.345602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:14.412959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c549d-06df-4d64-b12c-e42756d09748 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:14.416683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:14.417320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:14.417596 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:14.417870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:14.427164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:14.428652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:14.437895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1095/2202] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:20.597939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.678189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62df8-0283-45d2-aae8-74960819107b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.683074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.683873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.684150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.684424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] PATH_INFO: `/v3/users/ca54a9af332e48bbb53f20569cae49dd` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.685582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca54a9af332e48bbb53f20569cae49dd)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.688321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.839512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:20.852079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d66df0-774b-4326-af09-28d959965c0f service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:20.856959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:20.857595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:20.857876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:20.858699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] PATH_INFO: `/v3/users/176dc8fd7de44108a331ac50b57eb852` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:20.860038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176dc8fd7de44108a331ac50b57eb852)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:20.862028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:20.902983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca54a9af332e48bbb53f20569cae49dd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.905174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:20.915287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:20.926133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca54a9af332e48bbb53f20569cae49dd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.933733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca54a9af332e48bbb53f20569cae49dd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.934570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca54a9af332e48bbb53f20569cae49dd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:20.935235 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-58a64d43-bace-47e1-9fee-77ce56e2219f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca54a9af332e48bbb53f20569cae49dd'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.074506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1108/2203] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:20 2022] DELETE /identity/v3/users/ca54a9af332e48bbb53f20569cae49dd => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:21.079129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.079438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.079738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.100699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:21.101967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.161494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176dc8fd7de44108a331ac50b57eb852'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.163152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1109/2204] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:21.169704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176dc8fd7de44108a331ac50b57eb852'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.181107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.182588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.239742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51d190-46dc-4525-b358-aadcac06df1f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.243302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.243900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.244175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.244442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.253488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.254956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a6899-ac89-4cf1-843b-000a04d24745 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.264076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1110/2205] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:21.283270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176dc8fd7de44108a331ac50b57eb852'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.283918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176dc8fd7de44108a331ac50b57eb852'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.297559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176dc8fd7de44108a331ac50b57eb852'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.303763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1096/2206] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:20 2022] DELETE /identity/v3/users/176dc8fd7de44108a331ac50b57eb852 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:21.308393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.308681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.308954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.328295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:21.329418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.386486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1111/2207] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:21.396483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.397893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.455279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab3ca3-e2b2-46b1-8eea-1dec63da3ae1 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.459207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.459888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.460164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.460434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.471739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.473568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.485365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1097/2208] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:21.775715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.775715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.785375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ccd88-6c8d-4869-8a2b-734beea02f3f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.790286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:21.790955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:21.791233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:21.791518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] PATH_INFO: `/v3/projects/6a17d26e83d64847b5fa052862e6dde8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:21.792625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a17d26e83d64847b5fa052862e6dde8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:21.794655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:21.797430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:21.905835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a17d26e83d64847b5fa052862e6dde8', 'request_id': 'req-c3a7e23e-11ac-4869-be23-988ff8144be8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:21.975814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:21.977291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:21.990658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b20ef0-76a8-4864-af12-95d0ce8c785a service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:21.996757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a17d26e83d64847b5fa052862e6dde8', 'request_id': 'req-c3a7e23e-11ac-4869-be23-988ff8144be8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.005000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a17d26e83d64847b5fa052862e6dde8', 'request_id': 'req-c3a7e23e-11ac-4869-be23-988ff8144be8'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.018201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] The token cache is being invalidate because project 6a17d26e83d64847b5fa052862e6dde8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:22.018557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.019204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.020698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1112/2209] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:21 2022] DELETE /identity/v3/projects/6a17d26e83d64847b5fa052862e6dde8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:22.024639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.026137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.053242 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.053895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.054174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.054500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] PATH_INFO: `/v3/projects/848fcac123a34c39b9111d865e577e3c` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.055626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=848fcac123a34c39b9111d865e577e3c)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.057553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.092380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e23e-11ac-4869-be23-988ff8144be8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.096985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.097612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.097883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.098151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] PATH_INFO: `/v3/users/90f7c284a192423a960f80944c9b0a45` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.099298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f7c284a192423a960f80944c9b0a45)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.101202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.171227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:22.174741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848fcac123a34c39b9111d865e577e3c', 'request_id': 'req-cf409a6d-d985-4cd8-a598-35a6650925ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.180041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848fcac123a34c39b9111d865e577e3c', 'request_id': 'req-cf409a6d-d985-4cd8-a598-35a6650925ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.259995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848fcac123a34c39b9111d865e577e3c', 'request_id': 'req-cf409a6d-d985-4cd8-a598-35a6650925ec'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.273368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] The token cache is being invalidate because project 848fcac123a34c39b9111d865e577e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:22.273768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.274462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf409a6d-d985-4cd8-a598-35a6650925ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.276149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1098/2210] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:21 2022] DELETE /identity/v3/projects/848fcac123a34c39b9111d865e577e3c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:22.281014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa440-50cc-4a72-86e0-99749a78cdc7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.281340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa440-50cc-4a72-86e0-99749a78cdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.282532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23fa440-50cc-4a72-86e0-99749a78cdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.298623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f7c284a192423a960f80944c9b0a45'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.302818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a23fa440-50cc-4a72-86e0-99749a78cdc7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:22.304176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a23fa440-50cc-4a72-86e0-99749a78cdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.318603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f7c284a192423a960f80944c9b0a45'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.326630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f7c284a192423a960f80944c9b0a45'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.328643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f7c284a192423a960f80944c9b0a45'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.329277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f7c284a192423a960f80944c9b0a45'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:22.369074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1099/2211] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:22.382960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.383271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.383544 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.403583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:22.404779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.413722 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1100/2212] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:22.417631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.426346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda8f4b-2203-4a07-9f45-f78d7cb89e09 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.430725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1113/2213] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:22 2022] DELETE /identity/v3/users/90f7c284a192423a960f80944c9b0a45 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:22.430957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.431600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.431882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.432156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.432799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:22.433105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.434851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.450121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1101/2214] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:22.451681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.465942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:22.520073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d643c-55c2-45d2-839e-96245eb95888 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.525052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.525825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.526105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.526376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.527038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.529345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.647907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1114/2215] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:22 2022] POST /identity/v3/projects => generated 383 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:22.649523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.660656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a43aa-5df1-4b26-801c-e81a058f6225 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.729229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.729917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.730264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.730626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.731329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.733437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.829903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1102/2216] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:22 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:22.831167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.841554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa764a3-3fc9-4a01-a3f4-25d8cf4dd986 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.846653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.848177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.848464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.848729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.849376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.851003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.863044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.865164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1115/2217] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:22.865303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:22.869199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.881872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a0a82-e312-43b4-b2d1-adfa10408d6b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.886850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.887456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.888352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.888648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.889298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.891226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.895310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93962b0e-d32f-48a7-97a5-b57094398a1d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.902191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.903081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.903459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.903893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] PATH_INFO: `/v3/projects/f3cded48b4204bebb4f43eb62b53ce50` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.905586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1116/2218] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:22.907426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:22.915047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cded48b4204bebb4f43eb62b53ce50)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.917643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:22.919748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b045d6-eaca-489a-912e-ffce0e1d4bf3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:22.927808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:22.928624 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:22.928994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:22.929347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] PATH_INFO: `/v3/projects/c1ec83219f554689a00f6b501098da30/users/6a013a44a403409b97f13f58350fc781/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:22.946216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ec83219f554689a00f6b501098da30, user_id=6a013a44a403409b97f13f58350fc781, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:22.948925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e4da5-29d9-4801-9afa-fcb3b7b22918 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.031181 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1117/2219] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:22 2022] PUT /identity/v3/projects/c1ec83219f554689a00f6b501098da30/users/6a013a44a403409b97f13f58350fc781/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:23.033193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:23.040176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cded48b4204bebb4f43eb62b53ce50', 'request_id': 'req-57887e47-a295-4dd2-8b27-379715b37a66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.041622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30229-afac-4b1f-bef9-4d89cd7e3f56 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.041915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30229-afac-4b1f-bef9-4d89cd7e3f56 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.042192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30229-afac-4b1f-bef9-4d89cd7e3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.046041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cded48b4204bebb4f43eb62b53ce50', 'request_id': 'req-57887e47-a295-4dd2-8b27-379715b37a66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.070563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6dd30229-afac-4b1f-bef9-4d89cd7e3f56 None None] MFA Rules not processed for user `6a013a44a403409b97f13f58350fc781`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:23.071756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6dd30229-afac-4b1f-bef9-4d89cd7e3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.122468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cded48b4204bebb4f43eb62b53ce50', 'request_id': 'req-57887e47-a295-4dd2-8b27-379715b37a66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.135339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] The token cache is being invalidate because project f3cded48b4204bebb4f43eb62b53ce50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:23.135541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.136372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1118/2220] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:23.136505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:23.137668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1103/2221] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:22 2022] DELETE /identity/v3/projects/f3cded48b4204bebb4f43eb62b53ce50 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:23.142321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b7a60-c88d-4f5c-ae88-efb32bf95673 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.142640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b7a60-c88d-4f5c-ae88-efb32bf95673 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.142911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b7a60-c88d-4f5c-ae88-efb32bf95673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.148257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.149676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.162326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-460b7a60-c88d-4f5c-ae88-efb32bf95673 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:23.163638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-460b7a60-c88d-4f5c-ae88-efb32bf95673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.198632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:23.210295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57887e47-a295-4dd2-8b27-379715b37a66 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.213980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.214611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.214916 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.215187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.216890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.225134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1119/2222] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:23.228019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.229451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.233064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.233576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.233921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.239524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1104/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:23.255427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:23.256642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.264522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1120/2224] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:23.268867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.282835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa959f23-36e4-4328-95eb-c5235302e636 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.289346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.290025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.290371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.291357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.292023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:23.292330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.294398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.310579 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1105/2225] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:23.314254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.323642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1cfd7-3f2e-4073-9fe5-2860a7f753da None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.333411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.337815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.337815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.337815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.337815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.337815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.462495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1121/2226] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/projects => generated 409 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:23.463776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.474712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a07a8-1587-4254-a7b6-d1ec0646c2d2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.533864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.534506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.534784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.535051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.535692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.537600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.644287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1106/2227] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/users => generated 305 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:23.648313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.659319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d1c5f-eba3-4dcc-820a-d60565473a1e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.664074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.664704 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.665056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.665240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.665891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.667566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.678716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1122/2228] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:23.684279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.694676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77ee9-e6ef-4902-8af7-1b0761dd5a3a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.699346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.699945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.700220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.700493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.701135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.703530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8799e2-6716-46e1-809f-f0b512a3c0ce admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.717899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1107/2229] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:23.722917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:23.733108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850791f8-99c8-46b8-9567-f70bb18ec725 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:23.737596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:23.738352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.738666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.738943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] PATH_INFO: `/v3/projects/53ca1125170b4b57b785f60d28c9cc78/users/13b92eaec4994f63bf8030933c7c6634/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.753275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ca1125170b4b57b785f60d28c9cc78, user_id=13b92eaec4994f63bf8030933c7c6634, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:23.756116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:23.859481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1123/2230] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:23 2022] PUT /identity/v3/projects/53ca1125170b4b57b785f60d28c9cc78/users/13b92eaec4994f63bf8030933c7c6634/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:23.869956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:23.870238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:23.870593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:23.903305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] MFA Rules not processed for user `13b92eaec4994f63bf8030933c7c6634`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:23.903305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:23.933653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:23.992994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1108/2231] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:23.998634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.079673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434d2d7-c85b-4fe0-8794-49ff367a2ac6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.085500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.086109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.086385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.090791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.094290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.094290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.257830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1124/2232] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:23 2022] POST /identity/v3/projects => generated 407 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:24.262275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.274158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a865e-176c-4cc7-bfb3-d4b44ad9fee1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.342556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.343170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.343451 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.343718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.344396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.346271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.437561 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1109/2233] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:24 2022] POST /identity/v3/users => generated 310 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:24.443199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.454870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666840d-b77e-45b9-9902-04a8bbf1ae09 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.460175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.462664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.462664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.462664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.462664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.463511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.475121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1125/2234] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:24.479244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.497034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7e31a-746b-4452-9894-a0e5c7a6e4b3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.497034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.497034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.497034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.497684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] PATH_INFO: `/v3/projects/f110d5893c0349b88af5b58f28f1cf78/users/327ffa938f064b808b9f107680a7c9a2/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.512817 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f110d5893c0349b88af5b58f28f1cf78, user_id=327ffa938f064b808b9f107680a7c9a2, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.515861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.628252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1110/2235] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:24 2022] PUT /identity/v3/projects/f110d5893c0349b88af5b58f28f1cf78/users/327ffa938f064b808b9f107680a7c9a2/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:24.632315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.658287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4860987-233b-423c-9cf8-08d891f6ea9f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.731125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:24.745221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.745517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.745759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.746438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.747290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.750876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9539424-07bb-4668-b58c-1350205666da admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.761895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1126/2236] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:24.766352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:24.782902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890e7c8-1b62-4fed-97cf-e954ebe2961c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:24.787721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:24.788409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.788751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.789040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] PATH_INFO: `/v3/projects/f110d5893c0349b88af5b58f28f1cf78/users/327ffa938f064b808b9f107680a7c9a2/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.791274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f110d5893c0349b88af5b58f28f1cf78, user_id=327ffa938f064b808b9f107680a7c9a2, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:24.794131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:24.807709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1111/2237] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:24 2022] PUT /identity/v3/projects/f110d5893c0349b88af5b58f28f1cf78/users/327ffa938f064b808b9f107680a7c9a2/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:24.820951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:24.821249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:24.821511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:24.868108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] MFA Rules not processed for user `327ffa938f064b808b9f107680a7c9a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:24.869752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:24.954051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1127/2238] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:24 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:25.002737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:25.058469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8cdea-308d-4af8-a037-e2fb4666a872 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:25.062187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:25.062823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:25.063105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:25.063377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:25.072500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:25.073955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abe9ff-c211-4257-a552-4717e2fd4ce1 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:25.088867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1112/2239] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:25 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:26.056321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:26.057972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:26.073907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d6f5a-cbfe-4446-a944-d46d9f865059 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:26.078390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:26.079035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:26.079311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:26.079842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:26.159875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:26.161299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:26.170228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1128/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:26 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:33.691109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:33.691109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:33.691109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:33.692365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:33.693991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:33.717401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:33.719455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:33.738166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:33.783913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc7a5e-080a-4532-b72d-b41d9af7d2c7 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:33.789731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:33.790684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:33.790968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:33.792599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] PATH_INFO: `/v3/users/6a013a44a403409b97f13f58350fc781` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:33.792599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a013a44a403409b97f13f58350fc781)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:33.794345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1113/2241] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:31:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:33.799675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:33.822694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:33.858145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:33.859644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:33.967565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47a9f-1cb8-4850-b95d-defb9b3c2e04 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:33.970319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:33.970969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:33.971249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:33.971520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:33.980426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:33.982036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:33.991090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1114/2242] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:31:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:31:33.991237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a013a44a403409b97f13f58350fc781'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.018150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a013a44a403409b97f13f58350fc781'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.032800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a013a44a403409b97f13f58350fc781'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.033569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a013a44a403409b97f13f58350fc781'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.034299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6c7784b7-8746-4fa5-9af2-ccf04029c819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a013a44a403409b97f13f58350fc781'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.063756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:34.065313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.085393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40376cd8-efc6-4f5c-bbf3-2c4f9b2d21ea service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:34.091120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:34.091849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.092375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.092584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] PATH_INFO: `/v3/users/e119e37ecbc44b628e79b2c9459b2f8a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.093942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e119e37ecbc44b628e79b2c9459b2f8a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:34.096085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:34.129858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1129/2243] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:33 2022] DELETE /identity/v3/users/6a013a44a403409b97f13f58350fc781 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:34.134345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.134720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.135078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.172411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:34.174612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.289641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1130/2244] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:34.301725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:34.304092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.369070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e119e37ecbc44b628e79b2c9459b2f8a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.381641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e119e37ecbc44b628e79b2c9459b2f8a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.392742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5591d73-2f17-4ebc-bf57-1be14a7da6e3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:34.396726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:34.397599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.397894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.398225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.408870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:34.410574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97840702-01d5-4589-8e29-0ded18514369 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:34.426714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1131/2245] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:31:34.430720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e119e37ecbc44b628e79b2c9459b2f8a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.431616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e119e37ecbc44b628e79b2c9459b2f8a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.447206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e119e37ecbc44b628e79b2c9459b2f8a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:34.460034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1115/2246] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:34 2022] DELETE /identity/v3/users/e119e37ecbc44b628e79b2c9459b2f8a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:34.461199 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.461498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.461778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.487538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:34.488750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.522653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:34.551859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1132/2247] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:34.561930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:34.567221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.594566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:34.619744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cb1d7-aaed-4249-a382-46181c7db978 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:34.623319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:34.624468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.624771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.625024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.635902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:34.637625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:34.649004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1116/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:34.892321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:34.896472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:34.923543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ef0b6-8d44-4e2b-8510-0eb49b0b33f8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:34.929740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:34.935262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:34.935262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:34.935262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] PATH_INFO: `/v3/projects/c1ec83219f554689a00f6b501098da30` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:34.935262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ec83219f554689a00f6b501098da30)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:34.935262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:35.071076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ec83219f554689a00f6b501098da30', 'request_id': 'req-d14dc0b0-b517-45bf-87b7-5e81995bc23d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.150251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ec83219f554689a00f6b501098da30', 'request_id': 'req-d14dc0b0-b517-45bf-87b7-5e81995bc23d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.158179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:35.162659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ec83219f554689a00f6b501098da30', 'request_id': 'req-d14dc0b0-b517-45bf-87b7-5e81995bc23d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.163782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:35.177132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] The token cache is being invalidate because project c1ec83219f554689a00f6b501098da30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:35.177454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.181300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d14dc0b0-b517-45bf-87b7-5e81995bc23d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.181430 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72692d11-ac48-4997-b731-56048c3f2932 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:35.181534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1133/2249] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:34 2022] DELETE /identity/v3/projects/c1ec83219f554689a00f6b501098da30 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:35.264080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:35.264862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:35.265655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:35.265655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] PATH_INFO: `/v3/projects/065aa1d575f94b3e98f7d9e1e1e47ce9` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:35.266836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] RBAC: Authorizing `identity:delete_project(project_id=065aa1d575f94b3e98f7d9e1e1e47ce9)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:35.268854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:35.476442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065aa1d575f94b3e98f7d9e1e1e47ce9', 'request_id': 'req-5a89a537-7c12-4639-92d2-9e834077598c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.481756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065aa1d575f94b3e98f7d9e1e1e47ce9', 'request_id': 'req-5a89a537-7c12-4639-92d2-9e834077598c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.567383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065aa1d575f94b3e98f7d9e1e1e47ce9', 'request_id': 'req-5a89a537-7c12-4639-92d2-9e834077598c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.579296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] The token cache is being invalidate because project 065aa1d575f94b3e98f7d9e1e1e47ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:35.579584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.580232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-5a89a537-7c12-4639-92d2-9e834077598c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:35.581707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1117/2250] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:35 2022] DELETE /identity/v3/projects/065aa1d575f94b3e98f7d9e1e1e47ce9 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:35.586760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:35.587130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:35.587463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:35.607332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:35.609177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:35.668796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1134/2251] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:35.676683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:35.676983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:35.677284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:35.696960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:35.698175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:35.707124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1118/2252] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:35.711292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:35.721508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978f13e-0d5d-4950-b763-b8ffa4526501 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:35.726017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:35.726685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:35.726910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:35.727177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:35.727817 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:35.728113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:35.729693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:35.740869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1135/2253] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:35.752531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:35.761843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6079c75-b6ac-438d-945d-d16d453c1599 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:35.767063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:35.767657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:35.767937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:35.768208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:35.768846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:35.770882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:35.863193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1119/2254] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:35 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:35.875475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:35.903412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1f39d-68da-4fd7-89d0-e55132719f8a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:35.982863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:36.010942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:36.010942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.011384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.011384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.011948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:36.014042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:36.152035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1136/2255] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:35 2022] POST /identity/v3/users => generated 284 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:36.156403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:36.170007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4a035-36f2-474e-ba4d-621fad26f468 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:36.176260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:36.177196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.177552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.177889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.180220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:36.181002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:36.193507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1120/2256] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:36.197740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:36.214306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e6eb3-a6e0-4be7-b333-0cd067136938 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:36.219892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:36.220837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.221340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.221773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.222682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:36.224515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f76cdf-d4b4-4810-844a-997719344e7a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:36.244441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1137/2257] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:36.245570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:36.257361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478775b-acdb-48c9-8458-bb468bbf1e49 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:36.262216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:36.262912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.263194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.263468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] PATH_INFO: `/v3/projects/ffc5ec8973544fe9905cf394a0ae3aeb/users/06978aa617a847a385185e02608181e3/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.279542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc5ec8973544fe9905cf394a0ae3aeb, user_id=06978aa617a847a385185e02608181e3, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:36.282274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:36.285464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:36.407215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1121/2258] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:36 2022] PUT /identity/v3/projects/ffc5ec8973544fe9905cf394a0ae3aeb/users/06978aa617a847a385185e02608181e3/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:36.417822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.418105 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.418377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.447057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] MFA Rules not processed for user `06978aa617a847a385185e02608181e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:36.448351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:36.511243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1138/2259] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:36.522937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:36.528512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:36.583632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d17bf6-9fbe-421a-bd3e-04d4352b6afc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:36.587463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:36.588061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:36.588337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:36.588607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:36.596881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:36.598314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f0db4-d9d9-4206-b57c-06e917cf6f4a service glance] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:36.607729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1122/2260] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:40.913669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:40.915299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:40.995287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0814e04-d5e4-4eff-90fc-6b475b431647 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:41.001590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:41.002552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:41.003006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:41.003618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] PATH_INFO: `/v3/users/02dc36aabf204d6795f5b32254fef5ea` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:41.015076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02dc36aabf204d6795f5b32254fef5ea)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:41.015076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:41.282773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02dc36aabf204d6795f5b32254fef5ea'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:41.291474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:41.309086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02dc36aabf204d6795f5b32254fef5ea'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:41.343634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02dc36aabf204d6795f5b32254fef5ea'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:41.346012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02dc36aabf204d6795f5b32254fef5ea'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:41.347242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02dc36aabf204d6795f5b32254fef5ea'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:41.455157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1139/2261] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:40 2022] DELETE /identity/v3/users/02dc36aabf204d6795f5b32254fef5ea => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:41.461649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:41.462256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:41.462256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:41.506759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:41.508131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:41.673474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1123/2262] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:41.680268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:41.683117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:41.769066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c24d8c-3a14-4b1b-9f2b-1648973abdb2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:41.773736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:41.774385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:41.774803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:41.775084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:41.786669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:41.788333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68ab7-c5c7-42da-8ee4-c1d8390285ba service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:41.804653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1140/2263] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:42.517860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:42.519456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:42.541613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127d6c-6797-4f2d-963e-88335ee5af0f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:42.547658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:42.548674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:42.549351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:42.549935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] PATH_INFO: `/v3/projects/fec5165397344d70aecfdc82ca872557` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:42.559566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec5165397344d70aecfdc82ca872557)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:42.560532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:42.735853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec5165397344d70aecfdc82ca872557', 'request_id': 'req-a32a67b4-14eb-49d6-acde-34abf471b77c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:42.753321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec5165397344d70aecfdc82ca872557', 'request_id': 'req-a32a67b4-14eb-49d6-acde-34abf471b77c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:42.855747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec5165397344d70aecfdc82ca872557', 'request_id': 'req-a32a67b4-14eb-49d6-acde-34abf471b77c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:42.871524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] The token cache is being invalidate because project fec5165397344d70aecfdc82ca872557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:42.872074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:42.872938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:42.885988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1124/2264] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:42 2022] DELETE /identity/v3/projects/fec5165397344d70aecfdc82ca872557 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:43.051136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ae78-29cd-4fd6-9a57-27eea771ff59 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:43.051136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ae78-29cd-4fd6-9a57-27eea771ff59 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:43.051136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246ae78-29cd-4fd6-9a57-27eea771ff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:43.103730 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0246ae78-29cd-4fd6-9a57-27eea771ff59 None None] MFA Rules not processed for user `f4a01fadccd941fc98ecc99dac4e25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:43.105156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0246ae78-29cd-4fd6-9a57-27eea771ff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:43.212186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1141/2265] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:43.472219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:43.474120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:43.509200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:43.554798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a67b4-14eb-49d6-acde-34abf471b77c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:43.560298 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:43.561998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:43.562397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:43.562762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] PATH_INFO: `/v3/users/13b92eaec4994f63bf8030933c7c6634` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:43.563960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b92eaec4994f63bf8030933c7c6634)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:43.566217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:43.803691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b92eaec4994f63bf8030933c7c6634'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:43.814100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b92eaec4994f63bf8030933c7c6634'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:43.909757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b92eaec4994f63bf8030933c7c6634'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:43.910255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b92eaec4994f63bf8030933c7c6634'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:43.924555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b92eaec4994f63bf8030933c7c6634'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:43.932184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1125/2266] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:43 2022] DELETE /identity/v3/users/13b92eaec4994f63bf8030933c7c6634 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:43.939489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:43.940115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:43.940115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:43.961475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:43.962808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:44.042112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1142/2267] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:44.054093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:44.056350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:44.220765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dc946-7068-440d-a7b8-62af22a2ccfe admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:44.225755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:44.226499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:44.226860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:44.227216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:44.241927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:44.243826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622a882-ea49-4c84-a99f-553d5338af44 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:44.256370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1126/2268] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:44.779519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:44.782445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:44.805065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab8c1-ce95-45f7-a135-9ce333d6203c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:44.811793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:44.812989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:44.813432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:44.814130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] PATH_INFO: `/v3/projects/53ca1125170b4b57b785f60d28c9cc78` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:44.816781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ca1125170b4b57b785f60d28c9cc78)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:44.820021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:44.858000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:44.859028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:44.859454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:44.861218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:44.885051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] MFA Rules not processed for user `11770fbaeacd4fc7bc4052e83630dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:44.886496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:44.931666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:44.949488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1127/2269] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:44.985647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ca1125170b4b57b785f60d28c9cc78', 'request_id': 'req-d52bec67-c4c8-400c-8abb-cf6aff17be35'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.075774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ca1125170b4b57b785f60d28c9cc78', 'request_id': 'req-d52bec67-c4c8-400c-8abb-cf6aff17be35'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.083559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ca1125170b4b57b785f60d28c9cc78', 'request_id': 'req-d52bec67-c4c8-400c-8abb-cf6aff17be35'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.096390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] The token cache is being invalidate because project 53ca1125170b4b57b785f60d28c9cc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:45.096906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.097660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.099527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1143/2270] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:44 2022] DELETE /identity/v3/projects/53ca1125170b4b57b785f60d28c9cc78 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:45.104102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:45.118728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a5c9a-06b3-4d8f-9b46-c1e1b049593c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:45.186204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:45.187500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:45.187792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:45.188068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] PATH_INFO: `/v3/users/327ffa938f064b808b9f107680a7c9a2` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:45.189394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327ffa938f064b808b9f107680a7c9a2)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:45.191442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:45.351945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327ffa938f064b808b9f107680a7c9a2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.360091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327ffa938f064b808b9f107680a7c9a2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.446946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327ffa938f064b808b9f107680a7c9a2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.447592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327ffa938f064b808b9f107680a7c9a2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.465054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f6ab079-d768-43f3-a81b-c58082c2a444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327ffa938f064b808b9f107680a7c9a2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:45.471738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1128/2271] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:45 2022] DELETE /identity/v3/users/327ffa938f064b808b9f107680a7c9a2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:45.917847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:45.922843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:45.939366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bec67-c4c8-400c-8abb-cf6aff17be35 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.014080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.014832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.015112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.015385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] PATH_INFO: `/v3/projects/f110d5893c0349b88af5b58f28f1cf78` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.020710 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f110d5893c0349b88af5b58f28f1cf78)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.022798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.236030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f110d5893c0349b88af5b58f28f1cf78', 'request_id': 'req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.349785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f110d5893c0349b88af5b58f28f1cf78', 'request_id': 'req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.356340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f110d5893c0349b88af5b58f28f1cf78', 'request_id': 'req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.368583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] The token cache is being invalidate because project f110d5893c0349b88af5b58f28f1cf78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:46.368881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.369596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9c59b058-d6be-4a02-bdf9-1fadec3fc1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:46.371147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1144/2272] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:45 2022] DELETE /identity/v3/projects/f110d5893c0349b88af5b58f28f1cf78 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:46.376106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.381830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.381830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.397923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:46.399187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:46.462685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1129/2273] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:46.471022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.471417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.471692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.495333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:46.496576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:46.508601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1145/2274] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:31:46.511530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.522564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674fb44-601a-4c05-af23-9fca02bc05a8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.527685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.528386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.528737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.529145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.529824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:46.530261 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.532193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.535196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:46.546172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1130/2275] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:31:46.557495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.568527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ead3d-f27d-42fe-a5ed-55dffdaf9afb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.574538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.575155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.575681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.575966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.576673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.579333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.671979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1146/2276] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:46 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:46.677846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.693915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843319e-006d-4608-b862-ce253fe69b8b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.760065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.760785 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.761364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.761497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.762274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.764930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.857690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1131/2277] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:46 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:46.859663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.867724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:46.872499 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e0f27-adc1-423a-858e-5aeb52ff1ad7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.878705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.879676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.880041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.880438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.881264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.883444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.899635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1147/2278] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:46.901129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.920007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86266d-08b8-47c0-ab96-3351048e9a79 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.924065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.924899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.925230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.925588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.926346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.928339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a431b7c-e6ec-4f39-85c1-c616c84c7de4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:46.945666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1132/2279] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:46.952640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:46.965454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ee88e-3eda-49df-8bed-02f9f82c05c3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:46.972313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:46.972969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:46.973274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:46.973695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] PATH_INFO: `/v3/projects/a2aa021eeedb4f019a5a27d3a1881e4e/users/64a95b75cf264a53b80450da8bae5713/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:46.989929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2aa021eeedb4f019a5a27d3a1881e4e, user_id=64a95b75cf264a53b80450da8bae5713, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:46.992690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.078127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1148/2280] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:46 2022] PUT /identity/v3/projects/a2aa021eeedb4f019a5a27d3a1881e4e/users/64a95b75cf264a53b80450da8bae5713/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:47.095491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.095835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.096029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.129682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] MFA Rules not processed for user `64a95b75cf264a53b80450da8bae5713`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:47.131287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:47.210017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1133/2281] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:47.214899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.289884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd07598-9bec-4783-b362-e60fb641649a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.295167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.295809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.296155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.296438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.297186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.299158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.385839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1149/2282] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:47.390104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.407228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e9f036-c528-4103-96f7-816269826ca1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.479862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.480477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.480760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.481346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.481986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.485311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.498589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:47.643600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1134/2283] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:47.647236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.661019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fedd73-7b59-47dc-9eee-886cf8b0fdca admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.665877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.666524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.666806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.667089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.667840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.669436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.681048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1150/2284] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:47.686030 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.697031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fb43-dec2-4770-a1f6-9c12bb6949f6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.703057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.703724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.704015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.704404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] PATH_INFO: `/v3/projects/0479fdabe6a7409c8ba33321c943488e/users/63103114b2fd465f9af755c4c481c1df/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.720447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0479fdabe6a7409c8ba33321c943488e, user_id=63103114b2fd465f9af755c4c481c1df, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.723262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.793594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1135/2285] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:47 2022] PUT /identity/v3/projects/0479fdabe6a7409c8ba33321c943488e/users/63103114b2fd465f9af755c4c481c1df/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:47.797612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.808572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effd88b-0e0e-4b61-8236-987bfa29f4a2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.870955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.871719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.872053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.872329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.873121 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.874897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44802c-75a6-40e8-a874-7c170759810f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.891162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1151/2286] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:47.892859 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:47.904529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20a0b1-ef31-4f4c-8d3a-1df24104761b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:47.912291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:47.912978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.913484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.913605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] PATH_INFO: `/v3/projects/0479fdabe6a7409c8ba33321c943488e/users/63103114b2fd465f9af755c4c481c1df/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.915603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0479fdabe6a7409c8ba33321c943488e, user_id=63103114b2fd465f9af755c4c481c1df, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:47.918427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:47.930292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1136/2287] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:47 2022] PUT /identity/v3/projects/0479fdabe6a7409c8ba33321c943488e/users/63103114b2fd465f9af755c4c481c1df/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:47.940365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:47.940655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:47.940918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:47.975039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] MFA Rules not processed for user `63103114b2fd465f9af755c4c481c1df`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:47.979220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:48.027218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:48.068903 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1152/2288] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:48.083698 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:48.086110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:48.157705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede11d5-1a80-4f50-85e3-17581ece7ca6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:48.161855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:48.162486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:48.162764 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:48.163033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:48.175091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:48.177490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb70af-436b-4d0f-b5c4-c7b1c93e13b9 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:48.190847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1137/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:49.275661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.349404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d0ea4-a0a7-4331-b82a-9fbb1aca8cf4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.354750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.355383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.355658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.355932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] PATH_INFO: `/v3/users/64a95b75cf264a53b80450da8bae5713` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.357056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a95b75cf264a53b80450da8bae5713)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.359109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.514862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a95b75cf264a53b80450da8bae5713'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.527254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a95b75cf264a53b80450da8bae5713'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.539061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a95b75cf264a53b80450da8bae5713'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.540398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a95b75cf264a53b80450da8bae5713'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.540741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a95b75cf264a53b80450da8bae5713'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:49.645572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1153/2290] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:49 2022] DELETE /identity/v3/users/64a95b75cf264a53b80450da8bae5713 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:49.647758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.648312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.648550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.668932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:49.670388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.736589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1138/2291] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:49.746397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:49.748053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:49.799395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:49.815571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b38f-d141-43b0-9233-2a0453302eed admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:49.819331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:49.819911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:49.820320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:49.820586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:49.831269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:49.837005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:49.843275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1154/2292] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:50.558948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:50.560425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:50.580348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b800f2-b367-4c3f-97b0-f41e122e71a4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:50.596648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:50.596648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:50.596648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:50.596648 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] PATH_INFO: `/v3/projects/a2aa021eeedb4f019a5a27d3a1881e4e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:50.597075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2aa021eeedb4f019a5a27d3a1881e4e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:50.597075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:50.858618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2aa021eeedb4f019a5a27d3a1881e4e', 'request_id': 'req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:50.862374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:50.867854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2aa021eeedb4f019a5a27d3a1881e4e', 'request_id': 'req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:50.989597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2aa021eeedb4f019a5a27d3a1881e4e', 'request_id': 'req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.010142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] The token cache is being invalidate because project a2aa021eeedb4f019a5a27d3a1881e4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:51.010495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.011280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.013935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1139/2293] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:50 2022] DELETE /identity/v3/projects/a2aa021eeedb4f019a5a27d3a1881e4e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:51.017003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:51.027682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deca97-db8b-4a60-903f-3bc77922bda1 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:51.102161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:51.102739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:51.103024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:51.103447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] PATH_INFO: `/v3/users/63103114b2fd465f9af755c4c481c1df` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:51.105085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63103114b2fd465f9af755c4c481c1df)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:51.107050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:51.340787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63103114b2fd465f9af755c4c481c1df'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.356101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63103114b2fd465f9af755c4c481c1df'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.364289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63103114b2fd465f9af755c4c481c1df'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.365072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63103114b2fd465f9af755c4c481c1df'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.365822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd8f9a9-0cae-4009-97ab-8bfa90692d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63103114b2fd465f9af755c4c481c1df'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:51.464844 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1155/2294] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:31:51 2022] DELETE /identity/v3/users/63103114b2fd465f9af755c4c481c1df => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:51.884265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:51.888865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:51.921866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7727dc-1ea1-41a4-b311-eadc3b1f315c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:51.995575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:51.996110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:51.996391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:51.996669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] PATH_INFO: `/v3/projects/0479fdabe6a7409c8ba33321c943488e` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:51.997912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0479fdabe6a7409c8ba33321c943488e)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.000615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.122344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0479fdabe6a7409c8ba33321c943488e', 'request_id': 'req-d48a2b32-471f-4dea-b078-0bb40346596f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.129640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0479fdabe6a7409c8ba33321c943488e', 'request_id': 'req-d48a2b32-471f-4dea-b078-0bb40346596f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.189138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0479fdabe6a7409c8ba33321c943488e', 'request_id': 'req-d48a2b32-471f-4dea-b078-0bb40346596f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.202756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] The token cache is being invalidate because project 0479fdabe6a7409c8ba33321c943488e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:31:52.203154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.204088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d48a2b32-471f-4dea-b078-0bb40346596f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:31:52.205458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1140/2295] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:31:51 2022] DELETE /identity/v3/projects/0479fdabe6a7409c8ba33321c943488e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:52.213034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.213480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.213831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.239936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:52.241135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:52.321966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1156/2296] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:52.329552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.330036 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.330399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.352809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:52.354142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:52.363409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1141/2297] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:52.368061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.381667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2ff7e-b300-4279-99ae-074da8f8bf11 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.387137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.387747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.388052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.388331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.389251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:31:52.389377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.391003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.392734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:52.406458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1157/2298] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:31:52.411594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.422709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc5a5b-760e-4b96-8d2c-ad4a6f1d98df None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.427905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.428679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.428957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.429259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.429892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.432154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.499731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1142/2299] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/projects => generated 383 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:52.511035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.517615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aece97d-5cd6-4db8-bd4f-db22082c173e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.578066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.578702 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.578983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.579259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.579918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.581892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.670465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1158/2300] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:52.674548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.687076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f9fe7-f119-4d93-af03-0bda707d29f4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.692727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.693738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.693962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.694378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.695277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.697285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.712701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1143/2301] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:52.716714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.728046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefefd-8be6-43e4-a9f1-1de380eddd83 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.732884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.733548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.733885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.734239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.734979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.736635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5e0d7-d088-4f47-affb-af80969aa110 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.751899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1159/2302] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:31:52.753172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.765274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c026-e80c-4c6f-8a2c-75b3767df53b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:52.770589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:52.771315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.771659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.772014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] PATH_INFO: `/v3/projects/6a2b7155d5f54c5296c624e44d8c50d7/users/75fd0f6fa3a44848b23792b63850f4f4/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.791331 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a2b7155d5f54c5296c624e44d8c50d7, user_id=75fd0f6fa3a44848b23792b63850f4f4, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:52.794356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:52.854299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1144/2303] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:52 2022] PUT /identity/v3/projects/6a2b7155d5f54c5296c624e44d8c50d7/users/75fd0f6fa3a44848b23792b63850f4f4/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:52.866218 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:52.866533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:52.866807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:52.900726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] MFA Rules not processed for user `75fd0f6fa3a44848b23792b63850f4f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:52.902015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:52.972405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1160/2304] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:52.976902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:52.989041 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:53.056066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f299e-0ced-4a8b-9ad8-5617e3651584 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.062018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:53.062658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.062940 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.063217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.064032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:53.065978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:53.160537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1145/2305] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:52 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:53.161857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.174481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adafe85-3612-4fc7-8629-ab631246cf31 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.240675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:53.241449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.241736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.242003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.242842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:53.245845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:53.256015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:53.329440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1161/2306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:53 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:31:53.334014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.347849 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970791b-f6c0-469a-8be4-384213a68d11 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.353072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:53.353707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.353989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.354267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.355056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:53.356749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:53.377597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1146/2307] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:53.378664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.399193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9adf0e-fe93-42e8-ab70-cb6de78d803b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.405765 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:53.408756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.409060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.409340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.410690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:53.415484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b590aeb-c3dc-4b46-b078-0fd2211d90da admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:53.430287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1162/2308] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:53.434495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:53.456584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cf241-4fc1-4173-b161-446ced4bf7ed admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:53.472123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:53.473536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.474143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.475000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] PATH_INFO: `/v3/projects/66bbb4b60c35464bb25a2800c986bfcc/users/28942286b3924e0685d4da27850e9dc2/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.510238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bbb4b60c35464bb25a2800c986bfcc, user_id=28942286b3924e0685d4da27850e9dc2, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:53.514950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:53.608461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1147/2309] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:53 2022] PUT /identity/v3/projects/66bbb4b60c35464bb25a2800c986bfcc/users/28942286b3924e0685d4da27850e9dc2/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:31:53.610865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99cdd4-322d-4925-84d1-83dc077793e5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:53.611497 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99cdd4-322d-4925-84d1-83dc077793e5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:53.611783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99cdd4-322d-4925-84d1-83dc077793e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:53.721650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1c99cdd4-322d-4925-84d1-83dc077793e5 None None] MFA Rules not processed for user `28942286b3924e0685d4da27850e9dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:53.726328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1c99cdd4-322d-4925-84d1-83dc077793e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:53.894692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1163/2310] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:53.899522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.008206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af60b4e-1504-4de4-942b-35825cfd72c5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.015996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.016951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.017443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.017891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.018791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.026089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.097969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.097969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.098265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.107115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1148/2311] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:31:53 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:54.111588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.127066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:54.128280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.192795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:54.211846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69892116-06f9-429d-8407-ead5bff3505c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.218562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.219593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.220058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.220496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.221299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.224138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.237521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1164/2312] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:54.273969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.275558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:54.328887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1149/2313] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:31:54 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:31:54.333085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.351668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686afa77-67d8-4118-8768-e23df59c2a33 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.359231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.364011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.364011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.364011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.364011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.364011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.386918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1150/2314] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:54.395825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e969f-cc20-4357-913d-4b91d1a52551 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.403574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.403753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.403753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.403753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.403753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.412805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.414701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.422434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a44a0b-22a3-4cfe-9920-d89b038d5dd9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.439365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.441115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.441401 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.441825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] PATH_INFO: `/v3/projects/8fb460e54ca441d4a4e537820979e2a8/users/a0f6d5a0496d4f478aa8219ebe409ca5/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.451089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1165/2315] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:31:54.473885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb460e54ca441d4a4e537820979e2a8, user_id=a0f6d5a0496d4f478aa8219ebe409ca5, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.486889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.576853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1151/2316] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:54 2022] PUT /identity/v3/projects/8fb460e54ca441d4a4e537820979e2a8/users/a0f6d5a0496d4f478aa8219ebe409ca5/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:54.584933 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.595285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:31:54.601102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036b8c3-57ad-4075-a192-2bb56d74eded service swift] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:54.866096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:54.866754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:54.867045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:54.867611 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:54.867992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:54.880930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc317e8c-165f-4618-a862-dd073402a81b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:54.925983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1166/2317] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:31:54.930618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:54.959702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db188840-5c94-4251-80e8-bb0b6bd000d6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.004426 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.005155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.005484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.005812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] PATH_INFO: `/v3/projects/8fb460e54ca441d4a4e537820979e2a8/users/a0f6d5a0496d4f478aa8219ebe409ca5/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.025382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb460e54ca441d4a4e537820979e2a8, user_id=a0f6d5a0496d4f478aa8219ebe409ca5, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.028626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.060367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1152/2318] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:31:54 2022] PUT /identity/v3/projects/8fb460e54ca441d4a4e537820979e2a8/users/a0f6d5a0496d4f478aa8219ebe409ca5/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:31:55.083615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.097025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.097025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.189321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] MFA Rules not processed for user `a0f6d5a0496d4f478aa8219ebe409ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:31:55.189321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:55.474300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1167/2319] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:31:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:31:55.485848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:55.487602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:31:55.568047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d45b5d-0293-4b3e-bae7-578e1966b956 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:55.571883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:55.584372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:55.584372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:55.584372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:55.664819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:55.664819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:55.686972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1153/2320] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:31:58.192578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:31:58.217505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bf5b4-75ac-43a0-b45f-9b27b753a58e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:31:58.237776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:31:58.257623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:31:58.257623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:31:58.257623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:31:58.445308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:31:58.448706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bd49f-3c23-4a85-a29d-3c007458583e service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:31:58.531077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1168/2321] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:31:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:01.229602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:01.231370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:01.248794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a75e2c-ea0f-43c8-a40a-12166ecac359 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:01.260720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:01.261248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:01.261847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:01.262801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:01.278653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:01.283135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:01.309946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1154/2322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:03.572111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:03.572712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:03.572712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:03.625830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:03.627047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:03.684793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:03.780505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1169/2323] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:32:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:03.803291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:03.867290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:03.916822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb2c0-f798-40d3-ba5f-7fd926d091b6 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:03.921789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:03.922434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:03.922709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:03.922978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:03.931297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:03.933000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:03.941835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1155/2324] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:32:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:32:07.541226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:07.639741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e8ebb-4536-4b80-b440-2bbe97bc855b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:07.649160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:07.650056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:07.650344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:07.650667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] PATH_INFO: `/v3/users/75fd0f6fa3a44848b23792b63850f4f4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:07.652104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fd0f6fa3a44848b23792b63850f4f4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:07.655663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:07.894720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fd0f6fa3a44848b23792b63850f4f4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.911508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fd0f6fa3a44848b23792b63850f4f4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.921517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fd0f6fa3a44848b23792b63850f4f4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.922853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fd0f6fa3a44848b23792b63850f4f4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:07.936025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e463f74c-7e4b-4e5c-b625-b11ef9d9d248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fd0f6fa3a44848b23792b63850f4f4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.040267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1170/2325] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:07 2022] DELETE /identity/v3/users/75fd0f6fa3a44848b23792b63850f4f4 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:08.040378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.040809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.040809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.040809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.041069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.062184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426114c-1eaa-4cd2-a3c2-ab6cba68fd57 service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.064928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:08.066166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.118026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.119231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.119509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.119781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] PATH_INFO: `/v3/users/881062ec3efc4445a43e1ff772b09153` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.120920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=881062ec3efc4445a43e1ff772b09153)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.123098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.130764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:08.148372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1171/2326] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:08.158122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.159524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.223442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3a96a-e956-45ec-b247-c528aa14929e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.228422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.229038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.229359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.229632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.240347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.241802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.253465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1172/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:08.408576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881062ec3efc4445a43e1ff772b09153'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.416015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881062ec3efc4445a43e1ff772b09153'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.496905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881062ec3efc4445a43e1ff772b09153'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.498181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881062ec3efc4445a43e1ff772b09153'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.515205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881062ec3efc4445a43e1ff772b09153'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:08.526499 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1156/2328] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:08 2022] DELETE /identity/v3/users/881062ec3efc4445a43e1ff772b09153 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:08.757600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:08.769728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:08.800973 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f5340-54d5-45b3-abc2-2e4b549ad335 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:08.911994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:08.912681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:08.912961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:08.913234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] PATH_INFO: `/v3/projects/6a2b7155d5f54c5296c624e44d8c50d7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:08.918330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a2b7155d5f54c5296c624e44d8c50d7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:08.920341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:08.980006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.020729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.020729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d421f7e-cf9d-42c8-a9f3-61616ba18a4f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.020729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.021683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.021683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.021683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] PATH_INFO: `/v3/projects/1297d0249cf34d7290e867124be62377` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.021683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1297d0249cf34d7290e867124be62377)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.021683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.182385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a2b7155d5f54c5296c624e44d8c50d7', 'request_id': 'req-0fc14f33-4bbc-4f0c-a203-dfd529643263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.205068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.230522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1297d0249cf34d7290e867124be62377', 'request_id': 'req-b52550b9-b8b9-46b4-b4d2-669e95977f72'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.236896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1297d0249cf34d7290e867124be62377', 'request_id': 'req-b52550b9-b8b9-46b4-b4d2-669e95977f72'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.319454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a2b7155d5f54c5296c624e44d8c50d7', 'request_id': 'req-0fc14f33-4bbc-4f0c-a203-dfd529643263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.326883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a2b7155d5f54c5296c624e44d8c50d7', 'request_id': 'req-0fc14f33-4bbc-4f0c-a203-dfd529643263'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.346848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] The token cache is being invalidate because project 6a2b7155d5f54c5296c624e44d8c50d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:09.347140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.350296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.362551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1173/2329] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:08 2022] DELETE /identity/v3/projects/6a2b7155d5f54c5296c624e44d8c50d7 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:09.363943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.365353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.423492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:09.428290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1297d0249cf34d7290e867124be62377', 'request_id': 'req-b52550b9-b8b9-46b4-b4d2-669e95977f72'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.445636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] The token cache is being invalidate because project 1297d0249cf34d7290e867124be62377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:09.446547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.447605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b52550b9-b8b9-46b4-b4d2-669e95977f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.450557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1157/2330] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:08 2022] DELETE /identity/v3/projects/1297d0249cf34d7290e867124be62377 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:09.456143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca90a-9883-4210-b1b0-faeee9833783 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.456837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca90a-9883-4210-b1b0-faeee9833783 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.457542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ca90a-9883-4210-b1b0-faeee9833783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.477623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc14f33-4bbc-4f0c-a203-dfd529643263 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.519721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e13ca90a-9883-4210-b1b0-faeee9833783 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:09.520988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e13ca90a-9883-4210-b1b0-faeee9833783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.570310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.570991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.571434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.571717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] PATH_INFO: `/v3/users/28942286b3924e0685d4da27850e9dc2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.572944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28942286b3924e0685d4da27850e9dc2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.575470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.606962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1158/2331] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:09.609182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.609634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.609820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.632477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:09.633923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.655572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1159/2332] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:09.659841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.668891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ba68f-a58a-44c2-be29-a4cd588d047f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.673927 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.674542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.674814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.675081 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.675751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:09.676050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.677675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.690900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1160/2333] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:32:09.695115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.709608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b48b29-d2b4-47db-9107-e86acaddd545 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.714734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.715158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.715420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.715692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.716957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.719083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:09.767476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28942286b3924e0685d4da27850e9dc2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.780685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28942286b3924e0685d4da27850e9dc2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.787785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28942286b3924e0685d4da27850e9dc2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.788666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28942286b3924e0685d4da27850e9dc2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.789293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28942286b3924e0685d4da27850e9dc2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:09.841924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1161/2334] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:09.845899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:09.847609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:09.864976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1174/2335] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:09 2022] DELETE /identity/v3/users/28942286b3924e0685d4da27850e9dc2 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:09.918555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072c8be-5b04-47c9-8439-e461f7db9862 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:09.980868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:09.982152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:09.982553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:09.982832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:09.983504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:09.985474 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.048449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1162/2336] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:09 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:10.052689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.068798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122449d-22ad-4f5b-ae6a-9f3232e9546c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.073971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.074885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.075667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.076073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.076805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.078542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.093168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1175/2337] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:10.100547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.124131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada48be4-6bc7-466b-b918-a0ac232dd46d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.131029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.131666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.131948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.132223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.132986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.134727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ede0b7-90ac-4f52-98a7-5da07a54a1ce admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.151066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1163/2338] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:10.151743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.163586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0f007-7267-4d09-a965-09f68830758a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.168367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.169054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.169358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.169637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] PATH_INFO: `/v3/projects/e121652aac184961828460c8a9d22e1a/users/8cf8829f7d724bafa9f3ef3475ced84e/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.185091 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e121652aac184961828460c8a9d22e1a, user_id=8cf8829f7d724bafa9f3ef3475ced84e, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.187890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.270163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1176/2339] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:10 2022] PUT /identity/v3/projects/e121652aac184961828460c8a9d22e1a/users/8cf8829f7d724bafa9f3ef3475ced84e/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:32:10.281431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.281987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.282508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.316490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] MFA Rules not processed for user `8cf8829f7d724bafa9f3ef3475ced84e`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:10.318033 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.390828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1164/2340] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:10.392918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.395975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.400915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.405318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:10.479667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:10.479929 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4562c-3ed5-4cac-bead-a2e9b1372086 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.484996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.491437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.491437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.491437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] PATH_INFO: `/v3/projects/66bbb4b60c35464bb25a2800c986bfcc` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.491437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bbb4b60c35464bb25a2800c986bfcc)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.491437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.491935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bdae6-8a91-401f-9883-08e56ddc9c0b None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.493237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.493838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.494117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.494386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.503852 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.505288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:10.514516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1165/2341] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:10.595143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bbb4b60c35464bb25a2800c986bfcc', 'request_id': 'req-33c2b7f8-1d3b-4128-8340-188c2644a783'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.662530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:10.690831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bbb4b60c35464bb25a2800c986bfcc', 'request_id': 'req-33c2b7f8-1d3b-4128-8340-188c2644a783'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.697021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bbb4b60c35464bb25a2800c986bfcc', 'request_id': 'req-33c2b7f8-1d3b-4128-8340-188c2644a783'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.708489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] The token cache is being invalidate because project 66bbb4b60c35464bb25a2800c986bfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:10.708869 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.709584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:10.711172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1177/2342] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:10 2022] DELETE /identity/v3/projects/66bbb4b60c35464bb25a2800c986bfcc => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:10.715286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:10.726381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be209ea-f6d9-4542-bc04-131ff99fd9cf service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:10.781462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:10.782082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:10.782368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:10.782673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] PATH_INFO: `/v3/users/a0f6d5a0496d4f478aa8219ebe409ca5` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:10.783788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f6d5a0496d4f478aa8219ebe409ca5)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:10.785758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:11.001223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f6d5a0496d4f478aa8219ebe409ca5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.010590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f6d5a0496d4f478aa8219ebe409ca5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.107598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f6d5a0496d4f478aa8219ebe409ca5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.108266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f6d5a0496d4f478aa8219ebe409ca5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.120466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-8bacb2b3-c49f-4221-8933-af788aa43a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f6d5a0496d4f478aa8219ebe409ca5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:11.126464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1166/2343] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:10 2022] DELETE /identity/v3/users/a0f6d5a0496d4f478aa8219ebe409ca5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:11.498986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:11.921190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:11.923669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:11.936660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2b7f8-1d3b-4128-8340-188c2644a783 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:11.993760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:11.994320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:11.994703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:11.994974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] PATH_INFO: `/v3/projects/8fb460e54ca441d4a4e537820979e2a8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:11.996086 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb460e54ca441d4a4e537820979e2a8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:11.998051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.116953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:12.119717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb460e54ca441d4a4e537820979e2a8', 'request_id': 'req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.176308 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb460e54ca441d4a4e537820979e2a8', 'request_id': 'req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.182119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb460e54ca441d4a4e537820979e2a8', 'request_id': 'req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.193268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] The token cache is being invalidate because project 8fb460e54ca441d4a4e537820979e2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:12.193553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.194175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-51ce89e4-a974-46c8-8fa0-46e26ac6aee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:12.195683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1178/2344] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:11 2022] DELETE /identity/v3/projects/8fb460e54ca441d4a4e537820979e2a8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:12.200719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.201060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.201336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.221535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.222769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:12.281555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1167/2345] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:12.289138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.291433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.291433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.310719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.314520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:12.320370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1179/2346] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:12.324808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.335445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0c9d-1a3f-4fb2-977d-d2a6d2234130 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.340147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.341392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.341677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.341950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.342633 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:12.342937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.344583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.359111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1168/2347] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:12.361021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.371812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0675-5d65-4ad1-9e12-68803b5854a9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.377126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.377723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.378003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.378274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.379525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.381424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.441223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1180/2348] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:12.445355 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.456925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3bd3-34b1-421a-88d3-dd488ffb43b8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.498062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:12.515110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.515730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.515995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.516284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.516994 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.518872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.623378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1169/2349] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:12.627582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.644873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d700-be77-4f80-abe5-fac88e670c70 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.649642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.650246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.650556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.650830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.651603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.653803 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.664958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1181/2350] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:12.668960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.679447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4383a3-c0bb-4d31-8499-d611ca21dbca admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.684142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.684789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.685074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.685350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.686002 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.687756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d384a-dcbb-4c2f-ad7b-4bca274a4731 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.698704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1170/2351] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:12.703568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.718568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf02d0-0a90-4c04-92c3-2c0c87892fbe admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.719322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.719980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.720256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.720525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] PATH_INFO: `/v3/projects/0835e159896341d98201d9e553833d69/users/97f4dc3b263c43869df7bb3ad1d94e5c/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.734921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0835e159896341d98201d9e553833d69, user_id=97f4dc3b263c43869df7bb3ad1d94e5c, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:12.737558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:12.810782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1182/2352] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:12 2022] PUT /identity/v3/projects/0835e159896341d98201d9e553833d69/users/97f4dc3b263c43869df7bb3ad1d94e5c/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:12.820846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.821134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.821407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:12.849614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] MFA Rules not processed for user `97f4dc3b263c43869df7bb3ad1d94e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:12.850867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:12.918622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1171/2353] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:12.923370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:12.992523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397af9c-8aeb-4535-b7db-c0c7b36c8ae9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:12.998106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:12.998847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:12.999135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:12.999486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.007890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.007890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.127409 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1183/2354] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:12 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:13.131813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.142453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2461a6-aac9-4bda-8d11-5492a72d50ee None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.197204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.197804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.198088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.198366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.199270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.201354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.313977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1172/2355] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:13.317971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.328543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de28b53-6a4c-495d-9b16-be150dc39048 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.333248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.333974 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.334113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.334379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.335093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.336672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.348021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1184/2356] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:13.352063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.362882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9df8d-dd19-4631-be8d-863c9283ce84 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.373706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.374383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.374704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.374976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.375632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.377276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bce1c-dd4c-43da-bddf-67a8fdf157ef admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.387510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1173/2357] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:13.391411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.401130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe238f0c-0621-4bcb-b5ba-7cf87663f3ce admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.405659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.406301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.406608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.406881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] PATH_INFO: `/v3/projects/a0a07a78fbd349c7927f181c5d0489bf/users/2d7de679e21e4cd9a9b8241fafcd0e91/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.423767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a07a78fbd349c7927f181c5d0489bf, user_id=2d7de679e21e4cd9a9b8241fafcd0e91, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.426526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.509132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1185/2358] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:13 2022] PUT /identity/v3/projects/a0a07a78fbd349c7927f181c5d0489bf/users/2d7de679e21e4cd9a9b8241fafcd0e91/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:13.578680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.579225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.579344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.609621 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] MFA Rules not processed for user `2d7de679e21e4cd9a9b8241fafcd0e91`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:13.610901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:13.629234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.700855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1174/2359] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:13.705234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.726819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:13.767454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af3433-deea-46b3-9a3e-9950e5950d43 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.772137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.772796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.773076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.774170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.774170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.776010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:13.892257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1186/2360] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:13.896205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:13.907523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeeddf3-a300-4ca9-9c28-2bbcf8feaa1e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:13.971085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:13.971696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:13.971973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:13.972245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:13.972883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:13.974924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.090619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1175/2361] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:13 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:14.095714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.105822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6bcb-d8eb-4ad5-b625-cbe0e8ddcd0a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.110935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.111523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.111798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.112113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.112780 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.115046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.125956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1187/2362] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:14.129928 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.140790 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5d4bf-0296-40cc-85f0-34f4ff6d90b2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.145480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.146152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.146448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.146734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] PATH_INFO: `/v3/projects/4ade79a5c8cf48e1af54f2369531794d/users/91cfbab8630845fcb5b87531f1c7f393/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.162489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ade79a5c8cf48e1af54f2369531794d, user_id=91cfbab8630845fcb5b87531f1c7f393, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.166739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.233826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1176/2363] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:14 2022] PUT /identity/v3/projects/4ade79a5c8cf48e1af54f2369531794d/users/91cfbab8630845fcb5b87531f1c7f393/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:14.237784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.248372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113238f2-fea4-4e57-87bd-29539f954be3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.304822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.305433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.305705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.305976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.306650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.308246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb96d24-1b65-4e2e-8a1d-70d7663d599f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.318137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1188/2364] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:14.322299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.333762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24181c4-16d9-4657-9bab-b378538c6d70 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.338352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.339059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.339340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.339616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] PATH_INFO: `/v3/projects/4ade79a5c8cf48e1af54f2369531794d/users/91cfbab8630845fcb5b87531f1c7f393/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.341540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ade79a5c8cf48e1af54f2369531794d, user_id=91cfbab8630845fcb5b87531f1c7f393, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.344256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.354275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1177/2365] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:14 2022] PUT /identity/v3/projects/4ade79a5c8cf48e1af54f2369531794d/users/91cfbab8630845fcb5b87531f1c7f393/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:14.364436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.364721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.364992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.395149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] MFA Rules not processed for user `91cfbab8630845fcb5b87531f1c7f393`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:14.396367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:14.469990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1189/2366] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:14.481985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:14.491693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:14.567903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7feee-24b4-44fc-acb0-9e8c375413a6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:14.571763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:14.572378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:14.572683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:14.572957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:14.602162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:14.636914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:14.638475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:14.648160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1178/2367] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:16.835469 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:16.844996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f5db0-fc0c-4666-86c5-869923d9de9b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:16.848539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:16.849143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:16.849410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:16.849680 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:16.864591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:16.922143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:16.923711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45132e6d-ffaf-46aa-a4dd-484a73dae4ec service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:16.937168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1190/2368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:18.731857 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:18.733809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:18.750612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db42030f-be6f-47b5-9ee4-462f0b24c118 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:18.755932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:18.756547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:18.762792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:18.762792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:18.768051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:18.769598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22a716-65ff-4605-ba72-37a7eb311cd0 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:18.784433 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1179/2369] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:19.845574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:19.846169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:19.847569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:19.848950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1191/2370] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:32:19.855436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef6f5-61b9-4a86-ab62-7a5895bdb9d5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:19.856025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef6f5-61b9-4a86-ab62-7a5895bdb9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:19.856112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ef6f5-61b9-4a86-ab62-7a5895bdb9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:19.878262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9e8ef6f5-61b9-4a86-ab62-7a5895bdb9d5 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:19.879227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9e8ef6f5-61b9-4a86-ab62-7a5895bdb9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:20.005575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1180/2371] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:32:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:20.018834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.025965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:20.095598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53f131-8573-46d0-b18a-ffa5e4619439 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.099676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.100407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.100784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.101354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.113660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.115357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be830c-8ad6-4cf4-97c6-335f5d4360ce service glance] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.130184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1192/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:20.417131 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064816b1-c197-44fb-b358-18e629975b5d None None] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.417419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064816b1-c197-44fb-b358-18e629975b5d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.417690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064816b1-c197-44fb-b358-18e629975b5d None None] PATH_INFO: `/` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.419609 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1181/2373] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:32:20.424581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53da2-960f-411f-83e6-911bfb11f960 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.424980 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53da2-960f-411f-83e6-911bfb11f960 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.425320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53da2-960f-411f-83e6-911bfb11f960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.445097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-79f53da2-960f-411f-83e6-911bfb11f960 None None] MFA Rules not processed for user `1d00e1ef6c1a4f63a9a4514aac586c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:20.446541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-79f53da2-960f-411f-83e6-911bfb11f960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:20.502332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1193/2374] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:32:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:20.514052 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064816b1-c197-44fb-b358-18e629975b5d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.515762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-064816b1-c197-44fb-b358-18e629975b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:20.529711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064816b1-c197-44fb-b358-18e629975b5d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.533252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.533846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.534125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.534438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.541576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.543881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0a00c-4fd3-49b4-be47-eb20cd1c2f89 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.551973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1182/2375] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:20.849614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8812d40-c2a6-4a16-9cf5-9841d4312a78 None None] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.850085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8812d40-c2a6-4a16-9cf5-9841d4312a78 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.850204 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8812d40-c2a6-4a16-9cf5-9841d4312a78 None None] PATH_INFO: `/` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.851583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1194/2376] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 05:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 05:32:20.856894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.857335 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.857453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.877509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] MFA Rules not processed for user `a81fbda6880a41be8922a2f8978e7f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:20.878868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:20.889062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1183/2377] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 05:32:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:20.899685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8812d40-c2a6-4a16-9cf5-9841d4312a78 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:20.910969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8812d40-c2a6-4a16-9cf5-9841d4312a78 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:20.914524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:20.915115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:20.915394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:20.915672 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:20.926319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:20.927886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6430c4-cba3-41f8-8c0c-00535f8d3eab service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:20.938965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1195/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:24.653575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:24.655802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:24.754376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28d04c-4a65-4b69-b107-33465902fbe3 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:24.764202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:24.764833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:24.765168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:24.765438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:24.767639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:24.780513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:24.782140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:24.803045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1184/2379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:25.425249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:25.425712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:25.425805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:25.445060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:25.446263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:25.493252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:25.501192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1196/2380] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:32:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:25.513539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:25.514997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:25.529113 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:25.567373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:25.568895 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:25.582931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd0969-0399-4604-9f83-7debd038f2e9 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:25.588190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:25.588806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:25.589199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:25.589476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:25.598154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:25.599818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291f250f-b4f9-42ae-882c-b048a0755724 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:25.608922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1185/2381] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:32:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:32:25.640917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c098-7283-4ea6-b02d-8db612a49d9d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:25.646288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:25.649102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:25.649102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:25.649102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] PATH_INFO: `/v3/users/8cf8829f7d724bafa9f3ef3475ced84e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:25.649102 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf8829f7d724bafa9f3ef3475ced84e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:25.651022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:25.880020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf8829f7d724bafa9f3ef3475ced84e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:25.909272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf8829f7d724bafa9f3ef3475ced84e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:25.918880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf8829f7d724bafa9f3ef3475ced84e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:25.919905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf8829f7d724bafa9f3ef3475ced84e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:25.920743 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf8829f7d724bafa9f3ef3475ced84e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.036674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1197/2382] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:25 2022] DELETE /identity/v3/users/8cf8829f7d724bafa9f3ef3475ced84e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:26.038875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:26.039642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:26.040415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:26.061679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:26.063147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.145778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1186/2383] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:26.157402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:26.158889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.228139 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454884f-f2b5-4ca2-9dfa-5f30e56a8a05 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:26.237461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:26.238295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:26.238837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:26.239229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:26.250918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:26.252818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad4352-2efd-4c33-80f4-6a7439959752 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:26.296385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1198/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:26.334441 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:26.335959 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.357328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9a2b7-f48f-4874-ada2-58ae25fa3349 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:26.362925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:26.363719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:26.363851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:26.364115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] PATH_INFO: `/v3/users/97f4dc3b263c43869df7bb3ad1d94e5c` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:26.365280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f4dc3b263c43869df7bb3ad1d94e5c)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:26.367556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:26.594442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f4dc3b263c43869df7bb3ad1d94e5c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.620427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f4dc3b263c43869df7bb3ad1d94e5c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.747906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f4dc3b263c43869df7bb3ad1d94e5c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.748681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f4dc3b263c43869df7bb3ad1d94e5c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.764027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f4dc3b263c43869df7bb3ad1d94e5c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:26.775953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:26.776870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:26.777178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:26.786435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1187/2385] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:26 2022] DELETE /identity/v3/users/97f4dc3b263c43869df7bb3ad1d94e5c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:26.800715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:26.802190 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.820833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:26.871858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1199/2386] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:26.876717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:26.878843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.888116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:26.890063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:26.898668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09368d48-f680-4d06-b4d2-563ecaffab2c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:26.905375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:26.906369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:26.906793 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:26.907232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] PATH_INFO: `/v3/projects/e121652aac184961828460c8a9d22e1a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:26.909442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e121652aac184961828460c8a9d22e1a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:26.912117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:26.928896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:27.013585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9d50c-fbb1-4acc-aa1e-b1371acc62a3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.017266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.017883 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.018163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.018471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.034736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.036232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bab632-2ded-4553-9aa7-e6ed19217584 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.045148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e121652aac184961828460c8a9d22e1a', 'request_id': 'req-c56c7e9f-a4e5-4119-9451-9e01672fa053'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.046475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1200/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:27.061346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e121652aac184961828460c8a9d22e1a', 'request_id': 'req-c56c7e9f-a4e5-4119-9451-9e01672fa053'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.187921 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e121652aac184961828460c8a9d22e1a', 'request_id': 'req-c56c7e9f-a4e5-4119-9451-9e01672fa053'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.213871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] The token cache is being invalidate because project e121652aac184961828460c8a9d22e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:27.214293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.215189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c56c7e9f-a4e5-4119-9451-9e01672fa053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.217803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1188/2388] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:26 2022] DELETE /identity/v3/projects/e121652aac184961828460c8a9d22e1a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:27.223216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb910-083e-4f52-a195-229f8d70046a None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.223698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb910-083e-4f52-a195-229f8d70046a None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.224192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fb910-083e-4f52-a195-229f8d70046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.247055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-458fb910-083e-4f52-a195-229f8d70046a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:27.248358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-458fb910-083e-4f52-a195-229f8d70046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:27.396546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1201/2389] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:27.398460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.398774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.399051 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.421022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:27.422398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:27.438453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1189/2390] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:27.439662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fb910-083e-4f52-a195-229f8d70046a None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.452738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fb910-083e-4f52-a195-229f8d70046a None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.458100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.458772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.459048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.459321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.459993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:27.460284 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.461887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.476167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1202/2391] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:32:27.481316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.490885 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563d1ef-2a3a-404f-8c37-003d44111ad7 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.495728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.496892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.497173 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.497445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.498085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.500254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.575740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.586568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:27.610464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1190/2392] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:27 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:27.612124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71218-74fc-4090-bafc-f80ac8877a2b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.613593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.666340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:27.687521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.687521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.687521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.689057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] PATH_INFO: `/v3/projects/0835e159896341d98201d9e553833d69` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.689057 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0835e159896341d98201d9e553833d69)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.690879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.694435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbc419-f055-4c99-bc9a-5db08e3e7372 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.705275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.706149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.706647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.707014 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.707731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.709840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.822436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1191/2393] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:27 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:27.823566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.828562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:27.848227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd35a3-0b22-4c35-be33-1e3c897fdcd5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.858534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0835e159896341d98201d9e553833d69', 'request_id': 'req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.905193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.905741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.906016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.906286 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] PATH_INFO: `/v3/users/06978aa617a847a385185e02608181e3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.907489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06978aa617a847a385185e02608181e3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.909445 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:27.911097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:27.924955 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0835e159896341d98201d9e553833d69', 'request_id': 'req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.932276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0835e159896341d98201d9e553833d69', 'request_id': 'req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.967088 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] The token cache is being invalidate because project 0835e159896341d98201d9e553833d69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:27.967381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.968092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:27.969655 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1203/2394] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:27 2022] DELETE /identity/v3/projects/0835e159896341d98201d9e553833d69 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:27.970597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:27.971881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:27.984978 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe177c1-035e-4d2d-80ea-ab1bad86a2f3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:27.989824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:27.990451 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:27.990733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:27.991005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:27.991758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:27.993346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.007479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1204/2395] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:28.007479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:28.007479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.024244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd63de-1b85-47fe-a1ce-b7ead7d9fe17 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:28.030209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:28.030906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.031253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.031476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] PATH_INFO: `/v3/users/2d7de679e21e4cd9a9b8241fafcd0e91` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.032646 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7de679e21e4cd9a9b8241fafcd0e91)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:28.034606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.162791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06978aa617a847a385185e02608181e3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.169714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06978aa617a847a385185e02608181e3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.295875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06978aa617a847a385185e02608181e3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.296701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06978aa617a847a385185e02608181e3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.309402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06978aa617a847a385185e02608181e3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.315437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1192/2396] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:27 2022] DELETE /identity/v3/users/06978aa617a847a385185e02608181e3 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:28.316223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:28.317620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.351491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7de679e21e4cd9a9b8241fafcd0e91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.366065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7de679e21e4cd9a9b8241fafcd0e91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.376437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7de679e21e4cd9a9b8241fafcd0e91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.377179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7de679e21e4cd9a9b8241fafcd0e91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.377832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7de679e21e4cd9a9b8241fafcd0e91'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:28.410605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e242a1-a10b-4631-83b0-bbb8692f9d0c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:28.418580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:28.418580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.418580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.418580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.419393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:28.419906 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfac696-5d46-4102-bd8b-107de277306a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.431363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1193/2397] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:28.432744 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.433037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.433336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.454266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:28.455527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.468009 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1205/2398] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:28 2022] DELETE /identity/v3/users/2d7de679e21e4cd9a9b8241fafcd0e91 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:28.468235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1194/2399] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:28.468984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:28.470359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.479969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:28.481611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.523693 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:28.548684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e672eb4-e75a-4520-b36d-90713a543aee None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:28.552627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb3ff7-54d6-4388-9db6-69b53e3f6ec5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:28.559746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:28.560523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.560870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.561212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] PATH_INFO: `/v3/projects/be72f5c9d9724a2a87a0a97c0d2d0cd8/users/9d01f65308f544b79d9c65e57705d9b7/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.575787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:28.581153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be72f5c9d9724a2a87a0a97c0d2d0cd8, user_id=9d01f65308f544b79d9c65e57705d9b7, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:28.590433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.623026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:28.624096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.624519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.624960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.640190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:28.642690 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c6f3b1-df7c-4327-83bb-9db509cb9837 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.654731 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1195/2400] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:32:28.690843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1206/2401] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:28 2022] PUT /identity/v3/projects/be72f5c9d9724a2a87a0a97c0d2d0cd8/users/9d01f65308f544b79d9c65e57705d9b7/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:28.696971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.697255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.697545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.723706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:28.725299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:28.810796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1196/2402] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:28.827092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:28.907513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc21d-5637-4b88-8c5e-0929040e3844 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:28.913930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:28.924791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:28.924791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:28.924791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:28.927188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:28.928649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:28.940281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1207/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:29.132632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.134183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:29.149231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720076d3-9be8-467b-a0a5-89ef4655ecc2 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:29.154783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:29.156020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.157138 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.157472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] PATH_INFO: `/v3/projects/a0a07a78fbd349c7927f181c5d0489bf` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.158847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a07a78fbd349c7927f181c5d0489bf)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:29.160815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:29.187653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.189577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:29.217665 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ccd94b-6147-4c9e-8fcb-916d85a3abd1 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:29.232565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:29.236301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.236301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.236301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] PATH_INFO: `/v3/projects/ffc5ec8973544fe9905cf394a0ae3aeb` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.236301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc5ec8973544fe9905cf394a0ae3aeb)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:29.237925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:29.333531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a07a78fbd349c7927f181c5d0489bf', 'request_id': 'req-34cb8a57-2e94-4de5-a6e0-3a086211e21b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.340723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a07a78fbd349c7927f181c5d0489bf', 'request_id': 'req-34cb8a57-2e94-4de5-a6e0-3a086211e21b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.380625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc5ec8973544fe9905cf394a0ae3aeb', 'request_id': 'req-db8e945f-f8bb-48e7-bf96-362d2bbf0578'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.421960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a07a78fbd349c7927f181c5d0489bf', 'request_id': 'req-34cb8a57-2e94-4de5-a6e0-3a086211e21b'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.434986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] The token cache is being invalidate because project a0a07a78fbd349c7927f181c5d0489bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:29.435195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.435836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.439076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1197/2404] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:29 2022] DELETE /identity/v3/projects/a0a07a78fbd349c7927f181c5d0489bf => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:29.442889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.444449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:29.500620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:29.507522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc5ec8973544fe9905cf394a0ae3aeb', 'request_id': 'req-db8e945f-f8bb-48e7-bf96-362d2bbf0578'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.512877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb8a57-2e94-4de5-a6e0-3a086211e21b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:29.513825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc5ec8973544fe9905cf394a0ae3aeb', 'request_id': 'req-db8e945f-f8bb-48e7-bf96-362d2bbf0578'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.525085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] The token cache is being invalidate because project ffc5ec8973544fe9905cf394a0ae3aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:29.525377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.526043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-db8e945f-f8bb-48e7-bf96-362d2bbf0578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.528116 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1208/2405] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:29 2022] DELETE /identity/v3/projects/ffc5ec8973544fe9905cf394a0ae3aeb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:29.533912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736b0-8dd0-43da-a912-03b4f115c59f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.534196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736b0-8dd0-43da-a912-03b4f115c59f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.534501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b736b0-8dd0-43da-a912-03b4f115c59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.560076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-66b736b0-8dd0-43da-a912-03b4f115c59f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:29.561961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-66b736b0-8dd0-43da-a912-03b4f115c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:29.587866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:29.588628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.589294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.589470 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] PATH_INFO: `/v3/users/91cfbab8630845fcb5b87531f1c7f393` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.591370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] RBAC: Authorizing `identity:delete_user(user_id=91cfbab8630845fcb5b87531f1c7f393)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:29.594213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:29.621483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:29.643246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1209/2406] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:29.662359 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.662746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.663290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.712722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:29.713989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:29.732219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1210/2407] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:29.736309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.751487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a419ae-904d-4210-bf71-a1e2f76995ab None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:29.778662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:29.779744 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.780182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.780683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.781644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:32:29.782120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:29.784058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:29.801471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1211/2408] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:32:29.805887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.818262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be6792-1c92-47a7-adad-5b1ae3a935ee admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:29.824823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:29.825639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:29.825992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:29.826325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:29.827067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:29.827672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91cfbab8630845fcb5b87531f1c7f393'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.829288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:29.843970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91cfbab8630845fcb5b87531f1c7f393'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.967247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91cfbab8630845fcb5b87531f1c7f393'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.969185 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91cfbab8630845fcb5b87531f1c7f393'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:29.987172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1212/2409] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:29 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:32:29.991062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:29.997606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91cfbab8630845fcb5b87531f1c7f393'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:30.004855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1198/2410] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:32:29 2022] DELETE /identity/v3/users/91cfbab8630845fcb5b87531f1c7f393 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:30.100131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8acea3-d5e5-4936-99c0-e59e56eb294b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.191919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.192638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.192923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.193264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.193984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.196336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.311527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1213/2411] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:29 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:30.314141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.315820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:30.340575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6f6bc-627f-4374-9f4c-1480e607f38b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.352378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.353015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.353294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.353569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.354230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.359411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.385016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1199/2412] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:30.386724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.404419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ee196-c59c-48e4-9589-e5514f3cc62e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.411054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.411694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.412013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.412294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.412958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.426871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0fabe-da4a-4845-9889-69099e3c62a2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.441709 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1214/2413] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:30.443184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.455178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f638-7892-482f-b623-3760d2e273fc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.461281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.462256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.463176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.463678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] PATH_INFO: `/v3/projects/924b0db81eb1479e9bea0f8a60d66526/users/4e9d9a240b824236aa40a3d03922b51d/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.481306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=924b0db81eb1479e9bea0f8a60d66526, user_id=4e9d9a240b824236aa40a3d03922b51d, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.484171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e142a4df-5ef2-4ef9-ab18-b42cb314f51a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.599494 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1200/2414] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:30 2022] PUT /identity/v3/projects/924b0db81eb1479e9bea0f8a60d66526/users/4e9d9a240b824236aa40a3d03922b51d/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:30.604582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.604961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.605733 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.628156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:30.633695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:30.642273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.642273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.642273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.708750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:30.712125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] MFA Rules not processed for user `9d01f65308f544b79d9c65e57705d9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:30.713390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:30.760155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1215/2415] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:32:30.761054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.762987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:30.789238 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:30.812330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b5962b-1bda-4fbe-bc0c-bea316f6058b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.817612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.818474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.818866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.819223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] PATH_INFO: `/v3/projects/4ade79a5c8cf48e1af54f2369531794d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.820812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ade79a5c8cf48e1af54f2369531794d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.823452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.839069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1201/2416] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:30.840282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.841598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:30.909310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d241d6-457a-463e-8d32-8a7bcda08604 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:30.913269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:30.913964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:30.914344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:30.914742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:30.925595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:30.934679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:30.937389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ade79a5c8cf48e1af54f2369531794d', 'request_id': 'req-80f3ec3e-b6eb-47db-b2d6-5574382643ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:30.948556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1202/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:30.950537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:30.952371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:31.005033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ade79a5c8cf48e1af54f2369531794d', 'request_id': 'req-80f3ec3e-b6eb-47db-b2d6-5574382643ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:31.014397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ade79a5c8cf48e1af54f2369531794d', 'request_id': 'req-80f3ec3e-b6eb-47db-b2d6-5574382643ee'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:31.041440 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] The token cache is being invalidate because project 4ade79a5c8cf48e1af54f2369531794d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:32:31.041698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:31.046288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-80f3ec3e-b6eb-47db-b2d6-5574382643ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:32:31.048749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1216/2418] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:32:30 2022] DELETE /identity/v3/projects/4ade79a5c8cf48e1af54f2369531794d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:31.071516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a21139-f765-4e38-999b-ea25789974b9 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:31.159527 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:31.160194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:31.160570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:31.160846 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:31.163486 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:31.285620 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:31.323427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:31.323427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75733600-4cb0-4884-bea6-b0e06af0d207 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:31.324440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1203/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:34.008084 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:34.008752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:34.008752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:34.028662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] MFA Rules not processed for user `9d01f65308f544b79d9c65e57705d9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:34.029860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:34.036595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1217/2420] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:39.062321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.062891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.062891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.102492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] MFA Rules not processed for user `4e9d9a240b824236aa40a3d03922b51d`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:39.103721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:39.192157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1204/2421] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:39.201322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.205197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:39.303559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db66195-f2fb-440a-bce5-7fe05e206a19 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.308853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.309561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.309842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.310118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.310826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.312937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.412997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1218/2422] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:32:39 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:39.418292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.434148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfe814-c56a-486f-b529-6ca68d7085d4 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.514615 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.515431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.515799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.516171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.517170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.519247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.625266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1205/2423] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:32:39 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:32:39.626678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.640867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708fa0-0719-49d7-81f9-9575111064d3 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.646205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.646986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.647366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.647738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.648515 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.650217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.663436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1219/2424] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:39.667704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.678902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2ca62-2c7d-4482-ad1f-dfc59f0b2284 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.694771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.695779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.696059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.696339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] PATH_INFO: `/v3/projects/f1546b97662e445cb251eab3e976a888/users/74ddce1a80214a4687eb346b31b44fb4/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.715197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1546b97662e445cb251eab3e976a888, user_id=74ddce1a80214a4687eb346b31b44fb4, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.717936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.805140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1206/2425] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:39 2022] PUT /identity/v3/projects/f1546b97662e445cb251eab3e976a888/users/74ddce1a80214a4687eb346b31b44fb4/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:39.810736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.832629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbde3c-9479-48f1-9964-b09bd8a04b3e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.854326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:39.914390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.915452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.915950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.916441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.917397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.919348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5daa60-ab60-4215-b2a7-954294105104 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.943162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1220/2426] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:32:39.947660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:39.958898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36dab5-4df6-4797-adc4-dfa1675a936f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:39.964324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:39.965023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:39.965306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:39.965574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] PATH_INFO: `/v3/projects/f1546b97662e445cb251eab3e976a888/users/74ddce1a80214a4687eb346b31b44fb4/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:39.967674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1546b97662e445cb251eab3e976a888, user_id=74ddce1a80214a4687eb346b31b44fb4, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:39.970364 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:39.985932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1207/2427] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:32:39 2022] PUT /identity/v3/projects/f1546b97662e445cb251eab3e976a888/users/74ddce1a80214a4687eb346b31b44fb4/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:32:45.989862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697dcd-827c-4a51-87be-8b7b8c5cde55 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:45.990384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697dcd-827c-4a51-87be-8b7b8c5cde55 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:45.990384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34697dcd-827c-4a51-87be-8b7b8c5cde55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:46.020609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-34697dcd-827c-4a51-87be-8b7b8c5cde55 None None] MFA Rules not processed for user `74ddce1a80214a4687eb346b31b44fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:46.021847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-34697dcd-827c-4a51-87be-8b7b8c5cde55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:46.116723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1221/2428] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:46.128610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:46.130562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:46.165835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:46.199083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d9277-c0e4-4af3-8a9f-eaec021f818c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:46.202985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:46.203675 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:46.204027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:46.204369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:46.270557 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:46.272062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:46.282477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1208/2429] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:32:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:32:46.570148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49cc69-91fb-4366-9294-f693063ebf05 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:46.570376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49cc69-91fb-4366-9294-f693063ebf05 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:46.570695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49cc69-91fb-4366-9294-f693063ebf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:46.619608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8e49cc69-91fb-4366-9294-f693063ebf05 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:46.625420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8e49cc69-91fb-4366-9294-f693063ebf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:46.697322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1222/2430] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:32:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:46.723233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:46.723807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:46.818071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380acd2d-ccf3-4aa1-86c8-a24082f7770a service glance] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:46.824389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:46.825077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:46.825356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:46.825630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:46.834294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:46.836271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:46.844876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1209/2431] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:32:56.689635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=76 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:56.841027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=68 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:32:57.684053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4264a01-24da-42f9-bcf9-a622790c1990 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:57.684053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4264a01-24da-42f9-bcf9-a622790c1990 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:57.684053 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4264a01-24da-42f9-bcf9-a622790c1990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:57.708074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e4264a01-24da-42f9-bcf9-a622790c1990 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:57.709552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e4264a01-24da-42f9-bcf9-a622790c1990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:57.717959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1223/2432] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:32:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:32:57.732172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:32:57.743542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231aa28-f29e-4aec-84cf-f12b617a930c service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:32:57.752043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:32:57.752740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:57.753084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:57.753423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:57.762226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:32:57.762551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:32:57.762893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:32:57.765236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:32:57.767070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580501b-1e24-433f-afce-4099148ad2c4 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:32:57.778869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1210/2433] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:32:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:32:57.783810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] MFA Rules not processed for user `4e9d9a240b824236aa40a3d03922b51d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:32:57.785200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:32:57.793518 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1224/2434] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:32:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:33:03.411869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa45ec7-e686-42c1-a0e1-4289bdb35d87 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:03.412313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa45ec7-e686-42c1-a0e1-4289bdb35d87 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:03.412313 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa45ec7-e686-42c1-a0e1-4289bdb35d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:03.432141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2aa45ec7-e686-42c1-a0e1-4289bdb35d87 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:03.433349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2aa45ec7-e686-42c1-a0e1-4289bdb35d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:03.447734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1211/2435] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:33:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:03.459521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:03.472620 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caca72-7ee3-453a-991f-5e7e23f98776 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:03.477638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:03.478275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:03.478587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:03.478847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:03.487487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:03.489868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:03.499740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1225/2436] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:33:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:33:13.426263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=9 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:13.495532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=12 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:27.438104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09281b3-5e35-4e97-b71e-c531d066de0f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:27.438625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09281b3-5e35-4e97-b71e-c531d066de0f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:27.438828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09281b3-5e35-4e97-b71e-c531d066de0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:27.458044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f09281b3-5e35-4e97-b71e-c531d066de0f None None] MFA Rules not processed for user `11770fbaeacd4fc7bc4052e83630dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:27.459282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f09281b3-5e35-4e97-b71e-c531d066de0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:27.517625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1212/2437] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:33:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:37.511061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=25 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:40.776253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:33:40.777811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:40.836127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f34460-7329-4beb-8dfe-70f6d699abdc service swift] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:33:40.839777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:33:40.840367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:40.840642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:40.840918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:40.849594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:33:40.851083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d04cb3-ce71-43a1-b089-7d94aecb068f service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:33:40.860228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1226/2438] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:33:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:33:42.844112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05f23-6f26-4158-9585-7e1974683989 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:33:42.844703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05f23-6f26-4158-9585-7e1974683989 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:33:42.844703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05f23-6f26-4158-9585-7e1974683989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:33:42.864383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d3b05f23-6f26-4158-9585-7e1974683989 None None] MFA Rules not processed for user `f4a01fadccd941fc98ecc99dac4e25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:33:42.865605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d3b05f23-6f26-4158-9585-7e1974683989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:33:42.929873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1213/2439] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:33:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:33:50.856164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=26 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:33:52.922781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=25 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:01.234061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:01.234570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:01.234570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:01.254068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] MFA Rules not processed for user `f661527186794ba296b322c22a78f3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:01.255325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:01.309953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1227/2440] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:34:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:34:02.557402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:02.557904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:02.558008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:02.577122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] MFA Rules not processed for user `9d01f65308f544b79d9c65e57705d9b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:34:02.578328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:02.635867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1214/2441] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:34:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:34:11.303476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=25 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:12.627819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=25 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:14.872390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:14.874422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.886238 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05483b83-feeb-4595-a80e-0625e9e32bc9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:14.889828 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:14.890437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:14.890725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:14.890993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:14.892656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:14.960696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:14.962122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:14.972801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1228/2442] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:15.099637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:15.109657 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc5ed2-4a3f-4356-ac07-773ac8d6b58f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:15.113202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:15.113795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:15.114071 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:15.114337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:15.116086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:15.132555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:15.132555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:15.141091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1215/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:24.968712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=34 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:25.136339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:32.176350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:32.177781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:32.232414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc700adb-d2ef-47fc-ae7e-d4099d2ec0ba service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:32.236485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:32.237089 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:32.237365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:32.237634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:32.239446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:32.250720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:32.252125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:32.261004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1229/2444] 10.222.0.16 () {60 vars in 1396 bytes} [Thu Sep 1 05:34:32 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:37.762995 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:37.764435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:37.778576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89afa6e-6c8c-49b4-9ade-91b1b2b0dcd0 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:37.784837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:37.785438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:37.785714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:37.785990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:37.787815 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:37.799260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:37.800848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:37.809632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1216/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:42.256806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=26 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:45.589225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:34:45.594175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:45.609600 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d007186-247d-4684-8856-0b09c5442337 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:34:45.615294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:34:45.615912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:34:45.616188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:34:45.616455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:34:45.618213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:34:45.631556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:34:45.633385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:34:45.647207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1230/2446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:34:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:34:47.805503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:34:55.642147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:04.840644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:04.851901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbdd8-5a17-4114-96c5-952dd329e3dc service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:04.856096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:04.857005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:04.857638 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:04.858222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:04.862170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:04.954741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:04.956397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:04.969183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1217/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:35:08.963508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:08.965041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:09.043043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc22f-9fd4-4bf5-a7ba-3f76c59f3b2d service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:09.048742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:09.049374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:09.049652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:09.049923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:09.051740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:09.063027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:09.064959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:09.074641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1231/2448] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:09.126665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:09.128493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:09.141555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30367eac-db4f-4cea-b90e-3de4194f4621 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:09.146385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:09.147070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:09.147351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:09.147624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] PATH_INFO: `/v3/users/f4a01fadccd941fc98ecc99dac4e25c4` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:09.148734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a01fadccd941fc98ecc99dac4e25c4)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:09.150701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:09.394272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a01fadccd941fc98ecc99dac4e25c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:09.401151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a01fadccd941fc98ecc99dac4e25c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:09.493571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a01fadccd941fc98ecc99dac4e25c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:09.494207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a01fadccd941fc98ecc99dac4e25c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:09.506717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-0ab4c08d-60d9-44fd-a189-6ecd298ecbe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a01fadccd941fc98ecc99dac4e25c4'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:09.513215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1218/2449] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:35:09 2022] DELETE /identity/v3/users/f4a01fadccd941fc98ecc99dac4e25c4 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:09.791944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:09.803096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5485a7-cc7c-4984-b9f4-75b4b1c6ce22 service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:09.873589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:09.874162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:09.874465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:09.874740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] PATH_INFO: `/v3/projects/b34f5b5ce7c44930b3c85701d9ceb384` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:09.876384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b34f5b5ce7c44930b3c85701d9ceb384)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:09.878454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.005653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b34f5b5ce7c44930b3c85701d9ceb384', 'request_id': 'req-0df4396f-134b-43ba-b22f-93bc18afb00e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:10.136483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b34f5b5ce7c44930b3c85701d9ceb384', 'request_id': 'req-0df4396f-134b-43ba-b22f-93bc18afb00e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:10.146165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b34f5b5ce7c44930b3c85701d9ceb384', 'request_id': 'req-0df4396f-134b-43ba-b22f-93bc18afb00e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:10.158187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] The token cache is being invalidate because project b34f5b5ce7c44930b3c85701d9ceb384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:35:10.158505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:10.159369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0df4396f-134b-43ba-b22f-93bc18afb00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:10.160872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1232/2450] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:35:09 2022] DELETE /identity/v3/projects/b34f5b5ce7c44930b3c85701d9ceb384 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:10.165958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.166514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.166514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.186766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:10.188826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:10.249530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1219/2451] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:10.257891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.258176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.258478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.279113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:10.280405 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:10.288429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1233/2452] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:10.292892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.303800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782458b-5c52-4a3f-9cf4-b386549b9bb8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.308357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.308960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.309239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.309507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.310153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:35:10.310484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.312160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.314545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:10.325646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1220/2453] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:35:10.330774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.340401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484f435-868a-43b9-866a-1d551a1bba29 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.345176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.345809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.346085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.346358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.347034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.348937 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.361327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:10.492841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1234/2454] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:35:10.497101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.507198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed9f73-07e8-45d4-b992-20edca783637 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.563548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.564153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.564436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.564708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.565334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.567481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.716644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1221/2455] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:35:10.721363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.731576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680ad74-3f3f-4342-a872-aee17f1d168b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.737066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.737738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.738017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.743681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.743681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.743681 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.750658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1235/2456] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:35:10.754469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.764450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990458b4-2246-4b21-8b56-04e192aea678 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.769389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.770016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.770294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.770608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.771258 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.772993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90ce54-3eaf-43ba-800c-be9b3fb7444a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.784187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1222/2457] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:35:10.788323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:10.798760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565aee50-5964-420c-90e6-b3d073fb890e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:10.803429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:10.804133 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.804334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.804610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] PATH_INFO: `/v3/projects/fdee9665c45d41be9d34ba14def64757/users/9561f323e125411bb83ecce21bb085c0/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.819612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdee9665c45d41be9d34ba14def64757, user_id=9561f323e125411bb83ecce21bb085c0, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:10.821741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c84a-6cce-413c-b02c-9f4b47e538a6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.822026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c84a-6cce-413c-b02c-9f4b47e538a6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.822330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4c84a-6cce-413c-b02c-9f4b47e538a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.822464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:10.843367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e4d4c84a-6cce-413c-b02c-9f4b47e538a6 None None] MFA Rules not processed for user `11770fbaeacd4fc7bc4052e83630dbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:10.845199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e4d4c84a-6cce-413c-b02c-9f4b47e538a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:10.897812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1223/2458] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:35:10.958336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1236/2459] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:35:10 2022] PUT /identity/v3/projects/fdee9665c45d41be9d34ba14def64757/users/9561f323e125411bb83ecce21bb085c0/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:10.963257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:10.963548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:10.963823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:10.983467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:10.984694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.045485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1224/2460] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:35:11.055341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:11.057123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.115333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e7611-8ba7-4bcb-985a-4c5a88e08428 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:11.119068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:11.119673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:11.119983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:11.120292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:11.129216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:11.130667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:11.140256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1237/2461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:11.224978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:11.226393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.238257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7be3-05a9-4602-81ab-ebce65bb4f75 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:11.243971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:11.243971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:11.243971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:11.243971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:11.244721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.254705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:11.256129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:11.264125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1225/2462] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:11.798785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:11.800240 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.889462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9e81b-c067-49cd-817b-12d9748f38c7 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:11.913930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:11.914630 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:11.915127 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:11.915402 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:11.916270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:11.917488 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.918276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:11.931249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:11.997687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:12.000647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:12.002297 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:12.009360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b0ebc-a8eb-4018-a6a6-870985fa128e service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:12.015461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:12.016627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:12.016747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1238/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:12.016933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:12.017159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:12.018985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:12.030891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:12.032591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:12.043044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1226/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:15.778887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:15.792419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee64a9-fd50-4d35-95dc-01e1efe52905 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:15.799952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:15.800816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:15.801097 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:15.801366 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:15.803311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:15.822952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:15.863267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:15.863267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1239/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:16.045475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:16.047963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:16.068570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538b2d8-9650-4ec8-b359-bd2b1ed4f601 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:16.075732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:16.081535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:16.081535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:16.081535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] PATH_INFO: `/v3/users/9d01f65308f544b79d9c65e57705d9b7` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:16.081535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d01f65308f544b79d9c65e57705d9b7)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:16.081535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:16.336801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:16.352616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26627cc-3e29-4c5d-b8d2-639e1d67b770 service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:16.360166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d01f65308f544b79d9c65e57705d9b7'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:16.364078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:16.364706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:16.364993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:16.365266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:16.369039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:16.375570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d01f65308f544b79d9c65e57705d9b7'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:16.388008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:16.389918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:16.415855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1240/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:35:16.471028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d01f65308f544b79d9c65e57705d9b7'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:16.471680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d01f65308f544b79d9c65e57705d9b7'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:16.495395 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d3945f2f-1916-46af-ab5f-96355b3a055f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d01f65308f544b79d9c65e57705d9b7'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:16.509125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1227/2467] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:35:16 2022] DELETE /identity/v3/users/9d01f65308f544b79d9c65e57705d9b7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:17.189870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:17.215119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d876145-edc7-4bec-8be7-6ab38dc702f3 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:17.304841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:17.305490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:17.305976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:17.306431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] PATH_INFO: `/v3/projects/be72f5c9d9724a2a87a0a97c0d2d0cd8` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:17.307814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be72f5c9d9724a2a87a0a97c0d2d0cd8)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:17.309739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:17.459431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be72f5c9d9724a2a87a0a97c0d2d0cd8', 'request_id': 'req-89d85b76-226a-4874-9a30-2252fa3e744b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:17.572545 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be72f5c9d9724a2a87a0a97c0d2d0cd8', 'request_id': 'req-89d85b76-226a-4874-9a30-2252fa3e744b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:17.581552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be72f5c9d9724a2a87a0a97c0d2d0cd8', 'request_id': 'req-89d85b76-226a-4874-9a30-2252fa3e744b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:17.614491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] The token cache is being invalidate because project be72f5c9d9724a2a87a0a97c0d2d0cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:35:17.616034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:17.617565 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:17.619249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1241/2468] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:35:17 2022] DELETE /identity/v3/projects/be72f5c9d9724a2a87a0a97c0d2d0cd8 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:18.018887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b82ad-5689-4da5-8fd1-dca6f587d28d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:18.019437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b82ad-5689-4da5-8fd1-dca6f587d28d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:18.019971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b82ad-5689-4da5-8fd1-dca6f587d28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:18.099103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-290b82ad-5689-4da5-8fd1-dca6f587d28d None None] MFA Rules not processed for user `9561f323e125411bb83ecce21bb085c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:18.102543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-290b82ad-5689-4da5-8fd1-dca6f587d28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:18.180026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:18.190435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1228/2469] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:18.205960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:18.208042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:18.275098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:18.296381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d85b76-226a-4874-9a30-2252fa3e744b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:18.302062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:18.302750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:18.303095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:18.303368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:18.315914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:18.318495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:18.329407 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1242/2470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:20.994184 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:21.002550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:21.002550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:21.016421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] MFA Rules not processed for user `9561f323e125411bb83ecce21bb085c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:21.017678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:21.027651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1229/2471] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:35:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:22.100523 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:22.102047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:22.119141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634e5ab-ebeb-4de8-82ce-1c8186ffb312 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:22.139056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:22.139056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:22.139056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:22.139056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:22.139460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:22.155082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:22.157963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc62bec-536d-4677-8dba-07799005eead service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:22.172363 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1243/2472] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:25.237767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:25.254566 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e393c-f450-496d-a94b-bdee3f66a840 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:25.260530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:25.261296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:25.261574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:25.261845 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:25.263653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:25.276732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:25.283598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:25.289380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1230/2473] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:35:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:35:32.165984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=13 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:33.074584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903148d-66d7-4e2f-9418-7478ff819f2f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:33.074874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903148d-66d7-4e2f-9418-7478ff819f2f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:33.075150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903148d-66d7-4e2f-9418-7478ff819f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:33.095856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a903148d-66d7-4e2f-9418-7478ff819f2f None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:33.097090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a903148d-66d7-4e2f-9418-7478ff819f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:33.163487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1244/2474] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:33.204260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:33.207304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:33.276572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e092b-2c98-48cb-93b9-c00ee5c4c73c service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:33.284066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:33.284784 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:33.285392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:33.285847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:33.294787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:33.297101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:33.324816 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1231/2475] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:35:33.913104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:33.913104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:33.913104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:33.939459 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:33.941065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:33.948216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1245/2476] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:35:33.961608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:33.972530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ce0c6-10a5-4d9f-a4a8-cce16741752e service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:33.977756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:33.978358 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:33.978838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:33.979115 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:33.987865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:33.989475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:33.999126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1232/2477] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:35:43.931792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=28 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:43.995393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=49 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:44.456052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:44.461775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:44.539615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707ab17-3a19-4260-b026-188db7dea80f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:44.561325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:44.563340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:44.563812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:44.565157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] PATH_INFO: `/v3/users/4e9d9a240b824236aa40a3d03922b51d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:44.569538 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9d9a240b824236aa40a3d03922b51d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:44.573430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:44.795268 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9d9a240b824236aa40a3d03922b51d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:44.811618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9d9a240b824236aa40a3d03922b51d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:44.819948 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9d9a240b824236aa40a3d03922b51d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:44.820800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9d9a240b824236aa40a3d03922b51d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:44.821412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9d9a240b824236aa40a3d03922b51d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:44.884155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1246/2478] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:35:44 2022] DELETE /identity/v3/users/4e9d9a240b824236aa40a3d03922b51d => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:45.133977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:45.148179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82651f51-3bcc-48f9-95ff-16ab0b788c45 service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:45.219010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523683dc-0394-4949-8192-bae924187760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:45.219755 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523683dc-0394-4949-8192-bae924187760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:45.220110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523683dc-0394-4949-8192-bae924187760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:45.220455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523683dc-0394-4949-8192-bae924187760 admin admin] PATH_INFO: `/v3/projects/924b0db81eb1479e9bea0f8a60d66526` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:45.221684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523683dc-0394-4949-8192-bae924187760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924b0db81eb1479e9bea0f8a60d66526)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:45.223756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523683dc-0394-4949-8192-bae924187760 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:45.539676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924b0db81eb1479e9bea0f8a60d66526', 'request_id': 'req-523683dc-0394-4949-8192-bae924187760'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:45.547377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924b0db81eb1479e9bea0f8a60d66526', 'request_id': 'req-523683dc-0394-4949-8192-bae924187760'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:45.681989 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924b0db81eb1479e9bea0f8a60d66526', 'request_id': 'req-523683dc-0394-4949-8192-bae924187760'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:45.694548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] The token cache is being invalidate because project 924b0db81eb1479e9bea0f8a60d66526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:35:45.702802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:45.702802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:45.702802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1233/2479] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:35:45 2022] DELETE /identity/v3/projects/924b0db81eb1479e9bea0f8a60d66526 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:45.703109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:45.703109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:45.717104 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea5f36-93aa-4f12-9b88-7576678c1e10 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:45.775033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:45.775674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:45.775960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:45.776228 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] PATH_INFO: `/v3/users/74ddce1a80214a4687eb346b31b44fb4` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:45.777399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ddce1a80214a4687eb346b31b44fb4)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:45.779874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:45.802439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:46.001381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ddce1a80214a4687eb346b31b44fb4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.014385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ddce1a80214a4687eb346b31b44fb4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.022689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ddce1a80214a4687eb346b31b44fb4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.023718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ddce1a80214a4687eb346b31b44fb4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.024578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-eef79aaf-ffae-48c8-a071-519f0be464c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ddce1a80214a4687eb346b31b44fb4'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.102346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1247/2480] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:35:45 2022] DELETE /identity/v3/users/74ddce1a80214a4687eb346b31b44fb4 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:46.377569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:46.379135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:46.413570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523683dc-0394-4949-8192-bae924187760 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:46.479760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:46.480392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:46.480667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:46.480937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] PATH_INFO: `/v3/projects/f1546b97662e445cb251eab3e976a888` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:46.482176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1546b97662e445cb251eab3e976a888)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:46.484352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:46.602060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:46.617912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1546b97662e445cb251eab3e976a888', 'request_id': 'req-6e2437a9-dd72-4ef9-b244-b880f1bc5371'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:46.624170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1546b97662e445cb251eab3e976a888', 'request_id': 'req-6e2437a9-dd72-4ef9-b244-b880f1bc5371'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:47.039930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1546b97662e445cb251eab3e976a888', 'request_id': 'req-6e2437a9-dd72-4ef9-b244-b880f1bc5371'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:47.052022 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] The token cache is being invalidate because project f1546b97662e445cb251eab3e976a888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:35:47.052310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:47.052944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:35:47.054444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1234/2481] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:35:46 2022] DELETE /identity/v3/projects/f1546b97662e445cb251eab3e976a888 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:35:56.095935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:35:56.631776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4cdc-67f9-4914-8922-1625b046dd4d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:56.632306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4cdc-67f9-4914-8922-1625b046dd4d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:56.632607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a4cdc-67f9-4914-8922-1625b046dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:56.652551 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5a9a4cdc-67f9-4914-8922-1625b046dd4d None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:35:56.658108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5a9a4cdc-67f9-4914-8922-1625b046dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:56.710183 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1248/2482] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:35:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:35:56.722252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:35:56.723724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:35:56.792518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2437a9-dd72-4ef9-b244-b880f1bc5371 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:35:56.798053 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:35:56.798683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:35:56.798963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:35:56.799233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:35:56.807244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:35:56.808836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:35:56.817771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1235/2483] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:36:06.702711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=25 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:06.814424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=44,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:12.835277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:12.835727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:12.835826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:12.855559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:12.856784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:12.864684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1249/2484] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:12.876759 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:12.887320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d12688-9f3f-4db7-b4a2-c97cd50f6a51 service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:12.892199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:12.892783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:12.893056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:12.893320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:12.901236 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:12.902867 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:12.911687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1236/2485] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:36:22.849908 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:22.908129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:29.467734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:29.469732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:29.525619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735999c3-2d78-4f20-abae-cc1e407e4c8b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:29.529237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:29.529845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:29.530124 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:29.530394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:29.532727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:29.599954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:29.601309 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:29.611001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1250/2486] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:33.357848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:33.359838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:33.373978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb3770-a7ed-46e4-902d-1f799476c3c8 service swift] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:33.379466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:33.380594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:33.381199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:33.381477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] PATH_INFO: `/v3/users/11770fbaeacd4fc7bc4052e83630dbb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:33.382826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11770fbaeacd4fc7bc4052e83630dbb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:33.384853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:33.806795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11770fbaeacd4fc7bc4052e83630dbb3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:33.813942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11770fbaeacd4fc7bc4052e83630dbb3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:33.901162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11770fbaeacd4fc7bc4052e83630dbb3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:33.901712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11770fbaeacd4fc7bc4052e83630dbb3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:33.914547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ea52076f-378b-4934-b290-c63c8930c041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11770fbaeacd4fc7bc4052e83630dbb3'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:33.920442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1237/2487] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:33 2022] DELETE /identity/v3/users/11770fbaeacd4fc7bc4052e83630dbb3 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:34.165878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:34.176058 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac55af-e4e3-48dd-b9b5-44003a4b443c service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:34.233776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:34.234427 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.234766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.235042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] PATH_INFO: `/v3/projects/1a200065c13e42babe6353689aabd896` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.236171 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a200065c13e42babe6353689aabd896)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:34.238080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:34.364584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a200065c13e42babe6353689aabd896', 'request_id': 'req-337e50c7-e8d9-4371-a10c-989a118b7505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:34.367147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:34.449742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a200065c13e42babe6353689aabd896', 'request_id': 'req-337e50c7-e8d9-4371-a10c-989a118b7505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:34.456266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a200065c13e42babe6353689aabd896', 'request_id': 'req-337e50c7-e8d9-4371-a10c-989a118b7505'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:34.467752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] The token cache is being invalidate because project 1a200065c13e42babe6353689aabd896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:36:34.468039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:34.468684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-337e50c7-e8d9-4371-a10c-989a118b7505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:34.470243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1251/2488] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:36:34 2022] DELETE /identity/v3/projects/1a200065c13e42babe6353689aabd896 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:34.475120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.475999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.476273 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.498563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:34.500292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:34.559926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1238/2489] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:34.568556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.568845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.569123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.590415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:34.591662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:34.599816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1252/2490] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:36:34.604010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:34.615103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b4cd2-28ee-4d47-8aaf-e2df84070979 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:34.619871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:34.620503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.620782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.621054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.621705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:36:34.622009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:34.623663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:34.637094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1239/2491] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:36:34.640726 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:34.650037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc7ca2-932b-482d-b181-265b4554aba3 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:34.654762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:34.655394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.655673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.655944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.656570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:34.658478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:34.749826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1253/2492] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:34 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:34.753835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:34.764365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcde122-7527-4dd6-9747-ead6891199bf admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:34.821468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:34.822134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.822435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.822712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.823330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:34.825259 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:34.833440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:34.890078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1240/2493] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:36:34 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:34.894128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:34.904559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a88b0-005f-4dd9-bb8c-e29af3cdf89a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:34.909254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:34.909859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:34.910140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:34.910433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:34.911077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:34.912763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.045059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1254/2494] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:35.049647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.059825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff295ee-370e-41b9-b691-d7223f5ea9a4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.065016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.065603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.065877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.066144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.066805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.068407 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a8647-4c08-465f-8251-47abcb8945ba admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.079890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1241/2495] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:35.083865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.094132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8ce0-b633-4d82-af62-99bfa5b07b49 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.098797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.099434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.099711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.099982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] PATH_INFO: `/v3/projects/535a9eeaf0a840c3b7680b469a5e3fc2/users/5a2524dc602b46b58df8f961b9097fc8/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.114684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=535a9eeaf0a840c3b7680b469a5e3fc2, user_id=5a2524dc602b46b58df8f961b9097fc8, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.117323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.186211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1255/2496] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:35 2022] PUT /identity/v3/projects/535a9eeaf0a840c3b7680b469a5e3fc2/users/5a2524dc602b46b58df8f961b9097fc8/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:35.196334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.196618 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.196901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.224914 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] MFA Rules not processed for user `5a2524dc602b46b58df8f961b9097fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:35.226875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:35.289801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1242/2497] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:35.294244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.354918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e406c-f51c-4229-a00e-e05e750e2cef admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.360971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.361568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.361845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.362115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.362770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.364652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.457178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1256/2498] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:35 2022] POST /identity/v3/projects => generated 363 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:36:35.461204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.472489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e537bed-6f3a-439f-beb2-c5d1e3a20e45 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.527125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.527798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.528000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.528291 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.528908 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.531512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.633406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1243/2499] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:36:35 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:35.637467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.648546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebaaa6-efbb-42d6-a34f-1295b04a0505 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.653195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.653848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.654266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.654379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.654975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.657188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.667100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1257/2500] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:35.671822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.684611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce0a11-a012-47f8-9a6b-55ef6b93896b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.689397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.690056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.690337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.690645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] PATH_INFO: `/v3/projects/c3c994cd35534f0cadb6c100587969bf/users/c9c0db5b3aa44deaa1d9c8b73b6f270a/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.708034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c994cd35534f0cadb6c100587969bf, user_id=c9c0db5b3aa44deaa1d9c8b73b6f270a, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.710787 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.799114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1244/2501] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:35 2022] PUT /identity/v3/projects/c3c994cd35534f0cadb6c100587969bf/users/c9c0db5b3aa44deaa1d9c8b73b6f270a/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:36:35.805392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.815735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53536b8-b467-4bbd-be8b-34d32fb0c35b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.862295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:35.871893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.872492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.872773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.873051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.873684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.875854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a254c8-033a-41a0-ae53-8cd126929603 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.895354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1258/2502] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:36:35.895509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:35.900424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02879a60-3021-4818-a40e-3af34b4b7e6b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:35.905417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:35.906080 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.906360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.906667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] PATH_INFO: `/v3/projects/c3c994cd35534f0cadb6c100587969bf/users/c9c0db5b3aa44deaa1d9c8b73b6f270a/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.908612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c994cd35534f0cadb6c100587969bf, user_id=c9c0db5b3aa44deaa1d9c8b73b6f270a, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:35.911354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:35.922055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1245/2503] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:35 2022] PUT /identity/v3/projects/c3c994cd35534f0cadb6c100587969bf/users/c9c0db5b3aa44deaa1d9c8b73b6f270a/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:35.933884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:35.934252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:35.934569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:35.963345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] MFA Rules not processed for user `c9c0db5b3aa44deaa1d9c8b73b6f270a`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:35.969842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:36.029741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1259/2504] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:36.040924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:36.042603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:36.065028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:36.098628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae421c77-65c5-4a64-bf63-d8c8627b11be admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:36.102740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:36.103326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:36.103603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:36.103870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:36.112542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:36.113969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:36.123177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1246/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:36.332098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:36.341452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e89970-fbe6-4d10-a4cf-0a47a98325ea None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:36.344940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:36.345549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:36.345882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:36.346144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:36.401331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:36.402787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a090098-e85c-455d-99f2-76569c5d04c5 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:36.411387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1260/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:46.141866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=17 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:46.407573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=68 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:47.103783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:47.164760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d2f4d-f92d-4af5-8581-07766974964c service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:47.169318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:47.169924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:47.170200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:47.170500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] PATH_INFO: `/v3/users/5a2524dc602b46b58df8f961b9097fc8` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:47.171598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2524dc602b46b58df8f961b9097fc8)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:47.174148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:47.330556 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2524dc602b46b58df8f961b9097fc8'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:47.337681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2524dc602b46b58df8f961b9097fc8'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:47.404549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2524dc602b46b58df8f961b9097fc8'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:47.405197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2524dc602b46b58df8f961b9097fc8'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:47.418197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2524dc602b46b58df8f961b9097fc8'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:47.424768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1247/2507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:47 2022] DELETE /identity/v3/users/5a2524dc602b46b58df8f961b9097fc8 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:47.429153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:47.429690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:47.429690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:47.450340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:47.451720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:47.517330 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1261/2508] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:36:47.526955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:47.528349 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:47.584810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd10c-86c0-4495-ac80-56708bca380f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:47.588347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:47.588944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:47.589222 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:47.589495 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:47.599008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:47.600443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:47.609979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1248/2509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:48.102879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:48.104283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:48.117381 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c27541-3d43-4da8-af06-fd5370cb7f80 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:48.122768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:48.123436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:48.123717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:48.123991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] PATH_INFO: `/v3/projects/535a9eeaf0a840c3b7680b469a5e3fc2` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:48.125132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=535a9eeaf0a840c3b7680b469a5e3fc2)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:48.127141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:48.233188 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535a9eeaf0a840c3b7680b469a5e3fc2', 'request_id': 'req-9804a85f-104f-498c-8570-31163e70b038'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.306262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535a9eeaf0a840c3b7680b469a5e3fc2', 'request_id': 'req-9804a85f-104f-498c-8570-31163e70b038'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.312401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535a9eeaf0a840c3b7680b469a5e3fc2', 'request_id': 'req-9804a85f-104f-498c-8570-31163e70b038'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.323819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] The token cache is being invalidate because project 535a9eeaf0a840c3b7680b469a5e3fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:36:48.324108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.325036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.326468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1262/2510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:36:48 2022] DELETE /identity/v3/projects/535a9eeaf0a840c3b7680b469a5e3fc2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:48.331054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:48.341097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b18577-2290-4395-9289-931f03e6448b service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:48.359696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:48.396306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:48.397153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:48.397436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:48.397705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] PATH_INFO: `/v3/users/c9c0db5b3aa44deaa1d9c8b73b6f270a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:48.398904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c0db5b3aa44deaa1d9c8b73b6f270a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:48.400898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:48.563290 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c0db5b3aa44deaa1d9c8b73b6f270a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.570503 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c0db5b3aa44deaa1d9c8b73b6f270a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.661128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c0db5b3aa44deaa1d9c8b73b6f270a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.661692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c0db5b3aa44deaa1d9c8b73b6f270a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.674683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-562bcc16-1553-48c3-8856-814d6a462cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c0db5b3aa44deaa1d9c8b73b6f270a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:48.680585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1249/2511] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:48 2022] DELETE /identity/v3/users/c9c0db5b3aa44deaa1d9c8b73b6f270a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:49.050573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.052061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:49.065981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804a85f-104f-498c-8570-31163e70b038 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.123182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.123804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.124077 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.124343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] PATH_INFO: `/v3/projects/c3c994cd35534f0cadb6c100587969bf` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.126037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c994cd35534f0cadb6c100587969bf)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.127970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.227623 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:49.243436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c994cd35534f0cadb6c100587969bf', 'request_id': 'req-54333b27-fadf-42a7-8e3d-dd5077725035'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:49.346480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c994cd35534f0cadb6c100587969bf', 'request_id': 'req-54333b27-fadf-42a7-8e3d-dd5077725035'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:49.352540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c994cd35534f0cadb6c100587969bf', 'request_id': 'req-54333b27-fadf-42a7-8e3d-dd5077725035'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:49.365045 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] The token cache is being invalidate because project c3c994cd35534f0cadb6c100587969bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:36:49.365328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:49.366001 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54333b27-fadf-42a7-8e3d-dd5077725035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:49.367577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1263/2512] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:36:49 2022] DELETE /identity/v3/projects/c3c994cd35534f0cadb6c100587969bf => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:49.372523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.373040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.373040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.393856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:49.395086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:49.453017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1250/2513] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:49.460804 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.471936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.471936 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.483422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:49.485169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:49.492994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1264/2514] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:49.497078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.506538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790a5da-07dd-43d7-858a-57074ef53164 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.511189 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.511774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.512050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.512340 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.512961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:36:49.513260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.514891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.525980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1251/2515] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:36:49.530160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.539805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a9d88-a47d-4b2c-9f11-00fbf372ded4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.544395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.544985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.545293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.545562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.546205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.548712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.630915 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1265/2516] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:49 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:36:49.633156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.647045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c787f-f922-43ce-b3ff-13bd629bd8b8 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.674934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:49.709606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.710224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.710531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.710805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.711434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.713461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.802848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1252/2517] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:36:49 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:49.804323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.821774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3ed81-c453-4307-9420-645de45c0107 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.826941 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.827557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.827837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.828111 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.829798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.831279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.842631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1266/2518] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:49.847304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.858018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40557-ed68-41da-aa20-e8e71ea56ddc admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.863170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03414514-d349-4399-9316-75e0661f219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.863781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03414514-d349-4399-9316-75e0661f219c admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.864058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03414514-d349-4399-9316-75e0661f219c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.864333 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03414514-d349-4399-9316-75e0661f219c admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.864985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03414514-d349-4399-9316-75e0661f219c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.866644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03414514-d349-4399-9316-75e0661f219c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:49.878139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1253/2519] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:49.882622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:49.894275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6128fc6-022e-4564-ac73-a26851d28479 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:49.899205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:49.899930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:49.900242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:49.900512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] PATH_INFO: `/v3/projects/b4c076be1dd546faaee7fb5f3debac69/users/720c62abaa384321810b945b22090834/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:49.916442 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c076be1dd546faaee7fb5f3debac69, user_id=720c62abaa384321810b945b22090834, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:49.919159 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.027333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1267/2520] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:49 2022] PUT /identity/v3/projects/b4c076be1dd546faaee7fb5f3debac69/users/720c62abaa384321810b945b22090834/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:50.039220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.039501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.039770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.068212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] MFA Rules not processed for user `720c62abaa384321810b945b22090834`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:50.069500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:50.135612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1254/2521] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:50.140216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.204658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0062305-b79f-4cfd-aec9-067dad62536b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.209313 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.209911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.210185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.210473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.211114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.213078 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.341474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1268/2522] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:50 2022] POST /identity/v3/projects => generated 369 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:36:50.345694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.356616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfcb031-b240-480b-ba7a-3cdd4d907524 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.417319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.417946 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.418227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.418551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.419240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.421255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.528187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1255/2523] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:36:50 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:50.529647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.540514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f48e3-a6e3-46f1-b754-d611c2033ea1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.545258 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.552271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.552271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.552271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.552271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.552271 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.561306 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1269/2524] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:50.566090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.576693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece9122b-63c5-42ba-941f-dd2504691de3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.582133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.582809 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.583085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.583375 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] PATH_INFO: `/v3/projects/e5086e460b794379947275558804e687/users/4e4086c4834042d7bc826372941872e7/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.598976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5086e460b794379947275558804e687, user_id=4e4086c4834042d7bc826372941872e7, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.601613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.723568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.724006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1256/2525] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:50 2022] PUT /identity/v3/projects/e5086e460b794379947275558804e687/users/4e4086c4834042d7bc826372941872e7/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:50.736259 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2914b87-4841-40da-b457-33c23bdde085 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.759982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:50.824279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.824975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.825263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.825516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.826247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.828056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be20ffe-f411-414b-bfee-3ddb55f2c62d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.841788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1270/2526] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:50.846235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:50.852366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:50.860475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29843f2-87c2-411d-be01-ae604168d61f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:50.874820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:50.875821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.877432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.877432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] PATH_INFO: `/v3/projects/e5086e460b794379947275558804e687/users/4e4086c4834042d7bc826372941872e7/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.879803 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5086e460b794379947275558804e687, user_id=4e4086c4834042d7bc826372941872e7, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:50.882905 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:50.908966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1257/2527] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:50 2022] PUT /identity/v3/projects/e5086e460b794379947275558804e687/users/4e4086c4834042d7bc826372941872e7/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:50.910737 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:50.911107 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:50.911563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:50.958312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] MFA Rules not processed for user `4e4086c4834042d7bc826372941872e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:50.959918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:51.074559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1271/2528] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:51.083233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:51.085685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:51.166963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c674-1b8f-4a51-b9ef-ed04a04e99ce admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:51.171178 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:51.171792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:51.173357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:51.173357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:51.183162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:51.184892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:51.195894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1258/2529] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:51.311109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:51.312546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:51.325352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a60ab-c2c1-4021-8db1-5a8b33a2995f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:51.329021 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:51.329627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:51.329971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:51.330246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:51.404200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:51.405794 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:51.416167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1272/2530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:36:55.550695 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:55.552090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:55.608689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37f0b6-af9a-4c11-bbc0-9dd87ca3365a service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:55.612713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:55.613323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:55.613600 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:55.613868 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:55.615598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:55.684126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:55.685939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08fdafd-84e9-4b93-9c2c-6bbcf761e528 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:55.699211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1259/2531] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:56.339876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:56.351563 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47cbc6-6b4d-4a44-95f2-35e8e02e98bd service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:56.356492 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:56.357128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:56.357404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:56.357816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] PATH_INFO: `/v3/users/720c62abaa384321810b945b22090834` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:56.359022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=720c62abaa384321810b945b22090834)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:56.361082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:56.531119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:56.537905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720c62abaa384321810b945b22090834'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:56.551758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720c62abaa384321810b945b22090834'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:56.559024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720c62abaa384321810b945b22090834'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:56.559768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720c62abaa384321810b945b22090834'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:56.560375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720c62abaa384321810b945b22090834'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:56.641374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:56.641891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:56.641978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:56.645118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1273/2532] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:56 2022] DELETE /identity/v3/users/720c62abaa384321810b945b22090834 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:56.664999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:56.666237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:56.716511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:56.738289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1260/2533] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:56.749164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:56.751291 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:56.817131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06fdd4-e0ac-4649-8983-8baae6c32a8a admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:56.820942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:56.821550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:56.821825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:56.822094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:56.833631 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:56.835061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:56.845283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1274/2534] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:36:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:36:57.388839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:57.391508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:57.403799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9647cf1-9bff-4c8d-b9d7-e54cac3b837e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:57.409732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:57.410478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:57.410828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:57.411163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] PATH_INFO: `/v3/projects/b4c076be1dd546faaee7fb5f3debac69` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:57.412354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c076be1dd546faaee7fb5f3debac69)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:57.414327 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:57.527941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c076be1dd546faaee7fb5f3debac69', 'request_id': 'req-e05faa30-0d8b-4cf5-86fc-3b87b0514781'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.536737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c076be1dd546faaee7fb5f3debac69', 'request_id': 'req-e05faa30-0d8b-4cf5-86fc-3b87b0514781'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.596873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c076be1dd546faaee7fb5f3debac69', 'request_id': 'req-e05faa30-0d8b-4cf5-86fc-3b87b0514781'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.611789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] The token cache is being invalidate because project b4c076be1dd546faaee7fb5f3debac69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:36:57.612155 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.612743 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.615770 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1261/2535] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:36:57 2022] DELETE /identity/v3/projects/b4c076be1dd546faaee7fb5f3debac69 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:57.618673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:57.629157 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbe52c-4c78-4a35-be4c-ae10b8192423 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:57.695815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:57.696521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:57.696796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:57.697065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] PATH_INFO: `/v3/users/4e4086c4834042d7bc826372941872e7` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:57.698231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4086c4834042d7bc826372941872e7)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:57.700302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:57.895016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4086c4834042d7bc826372941872e7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.910543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:57.913092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4086c4834042d7bc826372941872e7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.920135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4086c4834042d7bc826372941872e7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.921056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4086c4834042d7bc826372941872e7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:57.922182 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-48d81b8c-2ec6-4984-af46-49d3b6425db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4086c4834042d7bc826372941872e7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.031232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1275/2536] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:57 2022] DELETE /identity/v3/users/4e4086c4834042d7bc826372941872e7 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:36:58.628122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:58.629894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:58.654086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05faa30-0d8b-4cf5-86fc-3b87b0514781 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:58.714060 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:58.715508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:58.715508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:58.715508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] PATH_INFO: `/v3/projects/e5086e460b794379947275558804e687` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:58.717712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5086e460b794379947275558804e687)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:58.718614 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:58.832514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5086e460b794379947275558804e687', 'request_id': 'req-cf5f9da3-07da-4535-861a-d14c55a98ea5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.838193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5086e460b794379947275558804e687', 'request_id': 'req-cf5f9da3-07da-4535-861a-d14c55a98ea5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.946151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5086e460b794379947275558804e687', 'request_id': 'req-cf5f9da3-07da-4535-861a-d14c55a98ea5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.949122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:58.971300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] The token cache is being invalidate because project e5086e460b794379947275558804e687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:36:58.971945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.972872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cf5f9da3-07da-4535-861a-d14c55a98ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:58.974988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1262/2537] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:36:58 2022] DELETE /identity/v3/projects/e5086e460b794379947275558804e687 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:58.979636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:58.984676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:58.984676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.008411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:59.010055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:59.106376 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1276/2538] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:59.114585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.115345 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.115644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.137672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:59.138986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:59.148396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1263/2539] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:36:59.152267 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.164163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3f5a-c6e6-4e0a-9dda-25d0480cfa1b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.169592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.170202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.170508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.170779 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.171636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:36:59.172095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.173952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.188285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1277/2540] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:36:59.192431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.214391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dec0d3-163a-4cec-8263-dc2f6ec33dfe None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.220190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.220872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.221219 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.221577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.223003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.224931 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.315123 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.317898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1264/2541] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:59 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:59.326028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b18527-0b21-4479-93a7-a5a240903384 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.395122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.395732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.396016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.396288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.396916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.398951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.519701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1278/2542] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:36:59 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:36:59.523661 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.541871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413d01f-4295-4750-9661-a1d53bf2263c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.546923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.547530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.547808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.548078 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.548748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.550468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.562229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1265/2543] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:59.566266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.576153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cdd9c-3e71-4a3d-868b-23fb5ffa4245 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.581311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.581956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.582175 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.582478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.583114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.584768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.596888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1279/2544] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:36:59.600179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.610646 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22ce48-50de-44f5-8981-576776c0bffb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.611358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.613329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:59.615696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.616356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.616636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.616909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] PATH_INFO: `/v3/projects/3d8ec041b5c84ac5b7ebdcf15ead8214/users/7885c2719e4642d1af83f8a9c5ac5dc6/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.631712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559f299-93aa-4bbd-9db6-d1edbe240566 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.636583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.637223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.637501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.637785 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] PATH_INFO: `/v3/users/f661527186794ba296b322c22a78f3f0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.638987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f661527186794ba296b322c22a78f3f0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.639104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8ec041b5c84ac5b7ebdcf15ead8214, user_id=7885c2719e4642d1af83f8a9c5ac5dc6, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.641475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.642749 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a0ac4-a214-4df6-a4b3-65e8c01fe722 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:36:59.716988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1266/2545] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:36:59 2022] PUT /identity/v3/projects/3d8ec041b5c84ac5b7ebdcf15ead8214/users/7885c2719e4642d1af83f8a9c5ac5dc6/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:36:59.728456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.728742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.729018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.755956 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] MFA Rules not processed for user `7885c2719e4642d1af83f8a9c5ac5dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:36:59.757169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:36:59.788738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:59.794295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f661527186794ba296b322c22a78f3f0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:59.806550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f661527186794ba296b322c22a78f3f0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:59.814234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f661527186794ba296b322c22a78f3f0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:59.815041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f661527186794ba296b322c22a78f3f0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:59.815688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f661527186794ba296b322c22a78f3f0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:36:59.821608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1267/2546] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:36:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:36:59.825990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:36:59.882688 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1280/2547] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:36:59 2022] DELETE /identity/v3/users/f661527186794ba296b322c22a78f3f0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:36:59.887417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08feab-5614-4991-bc1b-cb0cd0c11385 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:36:59.915160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:36:59.946571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:36:59.947539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:36:59.947828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:36:59.948112 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:36:59.948757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:36:59.950870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.038365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1268/2548] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:00.042283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.043809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.057281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ab2a6-ebfa-4a25-9842-59a1938ab348 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.113039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.113632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.113906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.114173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.114829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.116728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.142248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.144187 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.158204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d400496-10b3-4bb0-b7ec-5825bdffe0b9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.163040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.166632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.166632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.166632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] PATH_INFO: `/v3/projects/538589fdc80248a3a3e82802151b4eb6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.166632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=538589fdc80248a3a3e82802151b4eb6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.167489 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.220352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1281/2549] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:00.224212 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.233616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3ec86-3d00-4ca2-b53f-9682e7c89384 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.238661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.239225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.239498 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.239764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.240386 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.242031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.252659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1282/2550] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:00.256455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.265850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecfb73-0ad0-4538-8181-708631709653 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.270812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.275739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.275739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.275739 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] PATH_INFO: `/v3/projects/aeef7e722e694174a5eebaab4e396782/users/f538323a45d145639f0a99a7b8b3d014/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.284569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538589fdc80248a3a3e82802151b4eb6', 'request_id': 'req-be833cbf-5ef2-450d-9785-9a93074528d9'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:00.287119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeef7e722e694174a5eebaab4e396782, user_id=f538323a45d145639f0a99a7b8b3d014, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.289880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.290645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538589fdc80248a3a3e82802151b4eb6', 'request_id': 'req-be833cbf-5ef2-450d-9785-9a93074528d9'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:00.367758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1283/2551] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:00 2022] PUT /identity/v3/projects/aeef7e722e694174a5eebaab4e396782/users/f538323a45d145639f0a99a7b8b3d014/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:00.371572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.374263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538589fdc80248a3a3e82802151b4eb6', 'request_id': 'req-be833cbf-5ef2-450d-9785-9a93074528d9'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:00.387834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] The token cache is being invalidate because project 538589fdc80248a3a3e82802151b4eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:00.388177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:00.388842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-be833cbf-5ef2-450d-9785-9a93074528d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:00.390469 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1269/2552] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:00 2022] DELETE /identity/v3/projects/538589fdc80248a3a3e82802151b4eb6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:00.395075 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9281c-c3e7-4fe2-8c3f-355ae29875e5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.395419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9281c-c3e7-4fe2-8c3f-355ae29875e5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.395703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9281c-c3e7-4fe2-8c3f-355ae29875e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.415125 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e3e9281c-c3e7-4fe2-8c3f-355ae29875e5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:00.416350 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e3e9281c-c3e7-4fe2-8c3f-355ae29875e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.434069 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc37ce-a9e9-499c-92c4-ef495b9999f4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.439814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.440670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.440947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.441219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.441861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.443513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.453214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1284/2553] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:00.457362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.459025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.471146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc99be-91f0-4b81-9748-b44e7cbaca80 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.475764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.476388 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.476666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.476938 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] PATH_INFO: `/v3/projects/aeef7e722e694174a5eebaab4e396782/users/f538323a45d145639f0a99a7b8b3d014/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.477153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1270/2554] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:00.478877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeef7e722e694174a5eebaab4e396782, user_id=f538323a45d145639f0a99a7b8b3d014, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.481508 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcead081-b674-4db8-b881-3635d3827794 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.484316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:00.485328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.485483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.485734 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.494740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1285/2555] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:00 2022] PUT /identity/v3/projects/aeef7e722e694174a5eebaab4e396782/users/f538323a45d145639f0a99a7b8b3d014/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:00.506964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:00.508046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.508195 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.508335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.508604 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.536470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] MFA Rules not processed for user `f538323a45d145639f0a99a7b8b3d014`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:00.537734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.541016 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:00.570338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1271/2556] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:00.574705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.584511 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a09a1d-9bd8-44c7-93f4-97166f040758 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.589227 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.589961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.590263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.590608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.591277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:00.591696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.593408 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.608296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1272/2557] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:37:00.608477 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1286/2558] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:00.609750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.619691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.619840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d217e-ecaf-4305-a10b-a4bcd00fbec9 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.621138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:00.625101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.625703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.625981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.626288 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.626973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.628924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.680773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c508a-5128-414f-afc2-1d5d5643b2a8 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.684715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.685314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.685588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.685855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.695332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.696755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.707143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1287/2559] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:00.743526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1273/2560] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:00.747462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.762112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b4ee-d04f-43a5-9a7b-730814d5f325 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.782853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.829599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.830475 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.830601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.830854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.831473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.833383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.848073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51d754-adb6-4bd4-8b4c-1a17d71da7b6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.852824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.857953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.857953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.857953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] PATH_INFO: `/v3/users/7885c2719e4642d1af83f8a9c5ac5dc6` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.857953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7885c2719e4642d1af83f8a9c5ac5dc6)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.857953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.945932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1288/2561] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:00 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:00.949967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.959638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112a1f4-956e-4d9c-9571-6da49dd52d98 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.964470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.965042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.965319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.965590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.966222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:00.967923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:00.973234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:00.978178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1289/2562] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:00.982085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:00.992085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9075e31-8d7f-4562-a4cc-013984b073a5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:00.996608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:00.997167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:00.997445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:00.997721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:00.998337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.000595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.011094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1290/2563] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:01.015197 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.024713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f05c8-8ad5-435d-87d6-2d776e94b6ad admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.029335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.029985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.030326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.030632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] PATH_INFO: `/v3/projects/0584d9166a104a56aa0315e36ed2e993/users/8be5a1cda1a04602a18c5cb3c60f3a33/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.045901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0584d9166a104a56aa0315e36ed2e993, user_id=8be5a1cda1a04602a18c5cb3c60f3a33, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.049350 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd99a9-ddd3-42d9-9aaf-380c16cd5ae2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.052321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7885c2719e4642d1af83f8a9c5ac5dc6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:01.059779 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7885c2719e4642d1af83f8a9c5ac5dc6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:01.145068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7885c2719e4642d1af83f8a9c5ac5dc6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:01.145712 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7885c2719e4642d1af83f8a9c5ac5dc6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:01.151798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1291/2564] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:01 2022] PUT /identity/v3/projects/0584d9166a104a56aa0315e36ed2e993/users/8be5a1cda1a04602a18c5cb3c60f3a33/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:01.158707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-1fcd9bb6-fd01-45fb-9efe-cd6eab2b4204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7885c2719e4642d1af83f8a9c5ac5dc6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:01.161843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.162130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.162422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.164895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1274/2565] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:00 2022] DELETE /identity/v3/users/7885c2719e4642d1af83f8a9c5ac5dc6 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:01.170708 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.170993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.171760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.191565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:01.192249 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] MFA Rules not processed for user `8be5a1cda1a04602a18c5cb3c60f3a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:01.192962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.193410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.209488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:01.252729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1275/2566] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:01.259158 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1292/2567] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:01.262655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.263334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.264038 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.264628 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.277821 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e8d05-360b-4d51-bf33-2c003a18378b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.282879 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.283474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.283748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.284065 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.284682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.286601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.322130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f21a4-4595-4426-a0ef-119c762fdf1a None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.325762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.326370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.326671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.326970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.335680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.337162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.346883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1276/2568] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:01.375741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1293/2569] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:01 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:01.377356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.389264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c0fb0-12b8-4c73-9203-ac1a48bed2ae service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.452388 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.452992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.453264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.453532 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.454161 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.456122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.588886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1277/2570] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:01 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:01.590059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.600882 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f0705-482a-4fa1-b84a-cd993fe1e15e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.605561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.606147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.606434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.606713 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.607848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.609691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.621514 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1294/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:01.625569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.635957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b7e63-c1c6-41a5-91b3-6294565d1f06 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.644142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.644874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.645142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.645443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] PATH_INFO: `/v3/projects/63ab3687aec74cae8cac17c2418325f4/users/f8f8b07763c5432bab95351aa28a674e/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.662088 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ab3687aec74cae8cac17c2418325f4, user_id=f8f8b07763c5432bab95351aa28a674e, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.664871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.757159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1278/2572] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:01 2022] PUT /identity/v3/projects/63ab3687aec74cae8cac17c2418325f4/users/f8f8b07763c5432bab95351aa28a674e/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:01.761230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.771846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630bde9-8323-40f2-8937-7e0c9b7961f7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.816114 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.817964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.837422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.838023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.838355 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.838667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.839315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.840984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.852403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1295/2573] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:01.856344 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:01.859200 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:01.866590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460a78a-d731-44b2-8500-c0b10f6dc316 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.871534 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.872172 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.872444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.872712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] PATH_INFO: `/v3/projects/63ab3687aec74cae8cac17c2418325f4/users/f8f8b07763c5432bab95351aa28a674e/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.874640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=63ab3687aec74cae8cac17c2418325f4, user_id=f8f8b07763c5432bab95351aa28a674e, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.877247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f03b6da-911f-444a-ab6e-b4a40bc94f9a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:01.888317 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1296/2574] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:01 2022] PUT /identity/v3/projects/63ab3687aec74cae8cac17c2418325f4/users/f8f8b07763c5432bab95351aa28a674e/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:01.891167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c33dc9-d0b0-415c-9b39-7079c7505ddb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:01.900567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.900855 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.901129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.909870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:01.930784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] MFA Rules not processed for user `f8f8b07763c5432bab95351aa28a674e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:01.932352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:01.951727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:01.952922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:01.957406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:01.957406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] PATH_INFO: `/v3/projects/3d8ec041b5c84ac5b7ebdcf15ead8214` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:01.957406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8ec041b5c84ac5b7ebdcf15ead8214)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:01.957406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:02.001870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1297/2575] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:02.012977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:02.014326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:02.070559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292762f0-8942-41d1-b2c1-3bbd914d86ea None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:02.082939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8ec041b5c84ac5b7ebdcf15ead8214', 'request_id': 'req-946fcc51-3a58-4531-ad2d-cab76addea1a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.088847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8ec041b5c84ac5b7ebdcf15ead8214', 'request_id': 'req-946fcc51-3a58-4531-ad2d-cab76addea1a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.118550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:02.119257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:02.119537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:02.119800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:02.175564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:02.177712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:02.179263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:02.179801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8ec041b5c84ac5b7ebdcf15ead8214', 'request_id': 'req-946fcc51-3a58-4531-ad2d-cab76addea1a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.189050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1298/2576] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:02.192667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] The token cache is being invalidate because project 3d8ec041b5c84ac5b7ebdcf15ead8214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:02.193035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.194802 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.198386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1279/2577] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:01 2022] DELETE /identity/v3/projects/3d8ec041b5c84ac5b7ebdcf15ead8214 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:02.199482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:02.260916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6bea7-2231-494c-9128-b2888754ee94 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:02.265512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:02.266131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:02.266412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:02.266718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] PATH_INFO: `/v3/users/f538323a45d145639f0a99a7b8b3d014` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:02.267833 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f538323a45d145639f0a99a7b8b3d014)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:02.269845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:02.512408 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f538323a45d145639f0a99a7b8b3d014'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.524853 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f538323a45d145639f0a99a7b8b3d014'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.532244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f538323a45d145639f0a99a7b8b3d014'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.532921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f538323a45d145639f0a99a7b8b3d014'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.533529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f538323a45d145639f0a99a7b8b3d014'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:02.643654 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1299/2578] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:02 2022] DELETE /identity/v3/users/f538323a45d145639f0a99a7b8b3d014 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:03.055832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.057671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.071362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fcc51-3a58-4531-ad2d-cab76addea1a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.129545 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.130179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.130490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.130763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] PATH_INFO: `/v3/projects/aeef7e722e694174a5eebaab4e396782` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.131964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeef7e722e694174a5eebaab4e396782)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.133877 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.144092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:03.265772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeef7e722e694174a5eebaab4e396782', 'request_id': 'req-f866f127-0ba1-4d35-b8f4-140339db359a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.271299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeef7e722e694174a5eebaab4e396782', 'request_id': 'req-f866f127-0ba1-4d35-b8f4-140339db359a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.294357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.296076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.309748 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0290738-d0e8-4689-8c2d-d7ce519cb897 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.362384 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeef7e722e694174a5eebaab4e396782', 'request_id': 'req-f866f127-0ba1-4d35-b8f4-140339db359a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.366205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.366867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.367148 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.367423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] PATH_INFO: `/v3/users/8be5a1cda1a04602a18c5cb3c60f3a33` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.368638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be5a1cda1a04602a18c5cb3c60f3a33)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.370601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.375535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] The token cache is being invalidate because project aeef7e722e694174a5eebaab4e396782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:03.375828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.376481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f866f127-0ba1-4d35-b8f4-140339db359a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.377972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1280/2579] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:03 2022] DELETE /identity/v3/projects/aeef7e722e694174a5eebaab4e396782 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:03.382660 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774370-c6ee-40c3-b6d3-9ddd5f341d01 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.382925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774370-c6ee-40c3-b6d3-9ddd5f341d01 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.383194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5774370-c6ee-40c3-b6d3-9ddd5f341d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.387564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:03.403069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c5774370-c6ee-40c3-b6d3-9ddd5f341d01 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:03.404270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c5774370-c6ee-40c3-b6d3-9ddd5f341d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.412818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1281/2580] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:03.420370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.423961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.423961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.439854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:03.441048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.448969 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1282/2581] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:03.453352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.461874 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0d72f-2ff7-47d4-91ac-7b44e97e5753 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.466516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.467522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.467522 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.467732 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.468888 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:03.469198 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.470884 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.482264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1283/2582] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:37:03.486491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.496077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e223d-6ab9-434e-b702-725bcf0367e3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.500813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.501381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.501663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.501935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.502592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.504520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.565361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be5a1cda1a04602a18c5cb3c60f3a33'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.576756 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be5a1cda1a04602a18c5cb3c60f3a33'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.585223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be5a1cda1a04602a18c5cb3c60f3a33'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.585958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be5a1cda1a04602a18c5cb3c60f3a33'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.586595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-e21c36cc-b786-4d0e-9486-3bf9c679930d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be5a1cda1a04602a18c5cb3c60f3a33'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:03.627206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1284/2583] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:03 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:03.631429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.633059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.653753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1300/2584] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:03 2022] DELETE /identity/v3/users/8be5a1cda1a04602a18c5cb3c60f3a33 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:03.660008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.660296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.660598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.679703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:03.680920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.698303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77192aea-b4d1-46a2-913a-5387b8c4b448 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.740495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1301/2585] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:03.751146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.751537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:03.752992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:03.757678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.758300 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.758619 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.758889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.759720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.761653 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.810068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f365fd4-7e79-4161-8c0e-e7ad22c32d88 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.814273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.814909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.817015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.817015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.824700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.826115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.836428 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1302/2586] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:03.845870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1285/2587] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:03 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:03.847164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.857004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07980965-5ca9-46e9-a92d-a4056b6406d5 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.862364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.862985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.863260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.863529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.864160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.865745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.875913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1303/2588] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:03.880147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.890500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3b9b9-dac9-4ea3-8b97-b23fa35324b4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.895279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.895876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.896156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.896425 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.897072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.898711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c6ad6-0325-4509-8119-6526fe3ef512 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:03.909479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1286/2589] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:03.913731 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:03.924334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477acd-fc29-46d7-a3db-cb9f6cca5789 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:03.929615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:03.930262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:03.930643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:03.930914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] PATH_INFO: `/v3/projects/6d8b94c6caa34c178d85187b2147a437/users/6232623dd58e40008a756b6904e7a612/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:03.947472 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8b94c6caa34c178d85187b2147a437, user_id=6232623dd58e40008a756b6904e7a612, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:03.950180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:04.031552 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1304/2590] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:03 2022] PUT /identity/v3/projects/6d8b94c6caa34c178d85187b2147a437/users/6232623dd58e40008a756b6904e7a612/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:04.042801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:04.043094 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:04.043370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:04.075087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] MFA Rules not processed for user `6232623dd58e40008a756b6904e7a612`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:04.075087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:04.142710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1287/2591] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:04.153412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:04.155227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:04.161036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:04.213778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8095d9a-c923-4943-b1ee-8d9a1cd01af6 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:04.217429 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:04.218115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:04.218390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:04.218707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:04.226800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:04.228216 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:04.237335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1305/2592] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:04.278828 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:04.280819 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:04.352872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c3af8-5340-4ef7-9edc-76f451a4bcda None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:04.358012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:04.359386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:04.359664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:04.359935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] PATH_INFO: `/v3/projects/0584d9166a104a56aa0315e36ed2e993` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:04.361128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0584d9166a104a56aa0315e36ed2e993)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:04.363271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:04.577181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0584d9166a104a56aa0315e36ed2e993', 'request_id': 'req-a87f379e-f78e-4ae3-bdb1-4c393307be29'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:04.583083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0584d9166a104a56aa0315e36ed2e993', 'request_id': 'req-a87f379e-f78e-4ae3-bdb1-4c393307be29'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:04.768354 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0584d9166a104a56aa0315e36ed2e993', 'request_id': 'req-a87f379e-f78e-4ae3-bdb1-4c393307be29'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:04.770713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:04.780126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] The token cache is being invalidate because project 0584d9166a104a56aa0315e36ed2e993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:04.780416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:04.781044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:04.783213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1288/2593] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:04 2022] DELETE /identity/v3/projects/0584d9166a104a56aa0315e36ed2e993 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:04.787224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:04.797888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3096bc5-8d53-4ce9-b3c9-60c48963e2cb service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:04.854546 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:04.855178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:04.855455 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:04.855727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] PATH_INFO: `/v3/users/f8f8b07763c5432bab95351aa28a674e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:04.856901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f8b07763c5432bab95351aa28a674e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:04.859007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:05.027912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f8b07763c5432bab95351aa28a674e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.050916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f8b07763c5432bab95351aa28a674e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.050916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f8b07763c5432bab95351aa28a674e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.050916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f8b07763c5432bab95351aa28a674e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.050916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-174adf8c-6ba1-49c5-b956-3d0bc3d27739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f8b07763c5432bab95351aa28a674e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.116892 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1306/2594] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:04 2022] DELETE /identity/v3/users/f8f8b07763c5432bab95351aa28a674e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:05.580535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:05.582019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:05.596777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f379e-f78e-4ae3-bdb1-4c393307be29 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:05.656773 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:05.658716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:05.658716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:05.658716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] PATH_INFO: `/v3/projects/63ab3687aec74cae8cac17c2418325f4` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:05.659149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63ab3687aec74cae8cac17c2418325f4)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:05.661079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:05.802356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63ab3687aec74cae8cac17c2418325f4', 'request_id': 'req-a240b0eb-9601-4307-9039-bf9045733021'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.809757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63ab3687aec74cae8cac17c2418325f4', 'request_id': 'req-a240b0eb-9601-4307-9039-bf9045733021'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.894141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63ab3687aec74cae8cac17c2418325f4', 'request_id': 'req-a240b0eb-9601-4307-9039-bf9045733021'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.906709 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] The token cache is being invalidate because project 63ab3687aec74cae8cac17c2418325f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:05.907000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.907762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a240b0eb-9601-4307-9039-bf9045733021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:05.909529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1289/2595] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:05 2022] DELETE /identity/v3/projects/63ab3687aec74cae8cac17c2418325f4 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:05.914357 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:05.914914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:05.914914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:05.934912 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:05.936120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:05.954360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:05.997781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1307/2596] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:06.006093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.006667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.007174 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.028366 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:06.030015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:06.038182 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1290/2597] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:06.042586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.053049 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e800f-f911-4bc6-883c-514965c95b1d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.058154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.058787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.059063 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.059336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.059968 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:06.060382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.061960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599f5ff8-6d04-451a-92e9-ce92637355e8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.073757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1308/2598] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:37:06.078003 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.087207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38747a84-2c2a-49f6-849d-a980fd69f37d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.091913 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.092593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.092870 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.093141 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.094262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.096204 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.165548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.165835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.166109 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.185725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] MFA Rules not processed for user `6232623dd58e40008a756b6904e7a612`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:06.192156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:06.195602 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1291/2599] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/projects => generated 357 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:06.196679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.241433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1309/2600] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:06.256250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:06.260135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a43f50-e8a8-4304-8844-f728890493c5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.264904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.268403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.268403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.268403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.268403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.269190 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.383701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1292/2601] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/users => generated 279 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:06.384935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.396575 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9c714-848b-4955-a93c-b38a402e6c7f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.401247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.401981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.402257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.402566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.403205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.404788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.415128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1310/2602] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:06.427168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.438376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c68f9-d23d-4d3d-8822-a2787b602ee4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.443792 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.444398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.444676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.444949 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.445598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.447253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17a7ca-fb8b-4eb2-8407-387b25802869 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.457396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1293/2603] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:06.461735 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.472035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bad08f-ff32-4bc8-a5ca-1cafca7c28a4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.476763 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.482430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.482430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.482430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] PATH_INFO: `/v3/projects/5e7efb1dbfb54dc1a905f0449568663c/users/1b875c3b43834c0583a6b175ed1551e2/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.496154 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7efb1dbfb54dc1a905f0449568663c, user_id=1b875c3b43834c0583a6b175ed1551e2, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.499411 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.570921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1311/2604] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:06 2022] PUT /identity/v3/projects/5e7efb1dbfb54dc1a905f0449568663c/users/1b875c3b43834c0583a6b175ed1551e2/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:06.582597 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.583267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.583480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.615635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] MFA Rules not processed for user `1b875c3b43834c0583a6b175ed1551e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:06.616951 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:06.685446 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1294/2605] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:06.689897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.752028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c070d4-e1b8-465d-a11d-6bedf3c03b50 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.757819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.758517 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.758798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.759070 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.759696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.761663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:06.844845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1312/2606] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/projects => generated 355 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:06.849044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:06.861836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0051d149-041f-4bd7-b229-a98cb858d426 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:06.925758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:06.926367 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:06.926678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:06.926947 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:06.927651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:06.929567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.032193 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1295/2607] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:06 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:07.036207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.046387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e625861-6194-480e-b74a-a9eb961e44df admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.051035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.051622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.051902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.052169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.052806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.054434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.055635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:07.066122 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1313/2608] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:07.070665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.081644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169af6d-47f4-459c-959e-d3895abcbd4a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.086215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.087004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.087278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.087543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] PATH_INFO: `/v3/projects/244dc46c80544cadab0ab1b2846e14db/users/0a69d67d5a6e4adbb284a752758d754d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.104091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244dc46c80544cadab0ab1b2846e14db, user_id=0a69d67d5a6e4adbb284a752758d754d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.106854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.173745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1296/2609] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:07 2022] PUT /identity/v3/projects/244dc46c80544cadab0ab1b2846e14db/users/0a69d67d5a6e4adbb284a752758d754d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:07.177684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.187944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef44bc-8701-44ec-9b5b-1d977e0dbcb4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.245652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.246311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.246615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.246886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.247544 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.249126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fded49c-e289-48d6-94e7-3e4a3db5196d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.259177 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1314/2610] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:07.263490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.274179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b4f47-f416-41eb-963f-3b088c6c34d7 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.278834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.279473 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.281025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.281025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] PATH_INFO: `/v3/projects/244dc46c80544cadab0ab1b2846e14db/users/0a69d67d5a6e4adbb284a752758d754d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.281945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244dc46c80544cadab0ab1b2846e14db, user_id=0a69d67d5a6e4adbb284a752758d754d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.284610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.294458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1297/2611] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:07 2022] PUT /identity/v3/projects/244dc46c80544cadab0ab1b2846e14db/users/0a69d67d5a6e4adbb284a752758d754d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:07.304319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.309865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.309865 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.335369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] MFA Rules not processed for user `0a69d67d5a6e4adbb284a752758d754d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:07.336576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:07.403971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1315/2612] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:07.415369 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.417132 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:07.439705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:07.476886 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d4e53-e2cb-4a07-afef-a843c0dd9989 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.480665 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.481270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.481584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.481856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.491664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.493103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.503020 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1298/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:07.692251 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:07.701770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1804e0e1-5a7d-43c8-a2b1-7790aa896e22 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:07.705371 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:07.705964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:07.706243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:07.706622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:07.763160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:07.764627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eeba47-66fd-417b-9c71-48d8149638f1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:07.773608 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1316/2614] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:17.562181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=18 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:17.778581 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=93 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:24.999012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:25.011832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:25.012281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:25.012374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:25.052000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:25.053195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:25.105191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1024b5a-f3c8-4d62-9612-a619a1a15659 service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:25.109768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:25.110396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:25.110700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:25.110973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] PATH_INFO: `/v3/users/1b875c3b43834c0583a6b175ed1551e2` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:25.112104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b875c3b43834c0583a6b175ed1551e2)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:25.114039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:25.130888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1317/2615] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:25.314594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:25.316115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:25.375796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b875c3b43834c0583a6b175ed1551e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:25.384207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a8520-42db-4f93-a6be-c8464d748f6f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:25.384475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b875c3b43834c0583a6b175ed1551e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:25.389880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:25.390527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:25.390807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:25.391071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:25.399307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:25.400959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc14bc-3d6f-461a-ab45-e32fd2edc512 service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:25.409003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1318/2616] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:37:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:37:25.469769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b875c3b43834c0583a6b175ed1551e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:25.470385 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b875c3b43834c0583a6b175ed1551e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:25.483245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b875c3b43834c0583a6b175ed1551e2'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:25.489542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1299/2617] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:24 2022] DELETE /identity/v3/users/1b875c3b43834c0583a6b175ed1551e2 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:25.494125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:25.494426 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:25.494699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:25.534004 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:25.535245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:25.569850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:25.605360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1319/2618] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:25.615733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:25.617213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:25.673589 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:25.689280 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df469183-0b6c-4dfb-ac70-30bb4e48d713 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:25.693334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:25.698142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:25.698142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:25.698142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:25.705347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:25.706887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4c2e2-e50e-400f-afef-0882ea972fcb service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:25.717050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1300/2619] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:26.231663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:26.233410 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:26.247302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb50d7-ac44-4cab-b54b-b164335e308e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:26.254824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:26.255639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:26.256080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:26.256425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] PATH_INFO: `/v3/projects/5e7efb1dbfb54dc1a905f0449568663c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:26.264501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7efb1dbfb54dc1a905f0449568663c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:26.267153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:26.407256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7efb1dbfb54dc1a905f0449568663c', 'request_id': 'req-ee439642-91d7-41d4-a3ca-ab5f90580269'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.427418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:26.428015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:26.428172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:26.448225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:26.449357 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:26.463809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7efb1dbfb54dc1a905f0449568663c', 'request_id': 'req-ee439642-91d7-41d4-a3ca-ab5f90580269'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.470436 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7efb1dbfb54dc1a905f0449568663c', 'request_id': 'req-ee439642-91d7-41d4-a3ca-ab5f90580269'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.482046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] The token cache is being invalidate because project 5e7efb1dbfb54dc1a905f0449568663c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:26.482337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.483147 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.485166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1320/2620] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:26 2022] DELETE /identity/v3/projects/5e7efb1dbfb54dc1a905f0449568663c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:26.488800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:26.490202 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:26.511477 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1301/2621] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:37:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:26.525163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:26.526576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:26.564416 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee439642-91d7-41d4-a3ca-ab5f90580269 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:26.569265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:26.569901 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:26.570773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:26.570773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] PATH_INFO: `/v3/users/0a69d67d5a6e4adbb284a752758d754d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:26.571727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a69d67d5a6e4adbb284a752758d754d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:26.573661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:26.602137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5f533-e596-4b60-8cd7-77453ec74721 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:26.607284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:26.607896 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:26.608179 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:26.608452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:26.616799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:26.618393 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb0c2f-d683-4330-8ea8-75a0fbe17c51 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:26.627281 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1302/2622] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:37:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:37:26.890016 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a69d67d5a6e4adbb284a752758d754d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.900988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:26.902189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a69d67d5a6e4adbb284a752758d754d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.909020 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a69d67d5a6e4adbb284a752758d754d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.913658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a69d67d5a6e4adbb284a752758d754d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:26.913658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a69d67d5a6e4adbb284a752758d754d'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.034774 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1321/2623] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:26 2022] DELETE /identity/v3/users/0a69d67d5a6e4adbb284a752758d754d => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:27.384553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:27.384842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:27.385998 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:27.406181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:27.407833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:27.460703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:27.462354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:27.478582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1303/2624] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 05:37:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:27.509370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:27.511715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:27.541986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=100 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:27.553504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033419d-e135-49e9-a654-4b469c0cf12f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:27.558738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:27.559510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:27.559870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:27.560275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] PATH_INFO: `/v3/projects/244dc46c80544cadab0ab1b2846e14db` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:27.561860 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244dc46c80544cadab0ab1b2846e14db)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:27.564013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:27.607628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1552-cfc7-4e8b-b378-19df8f70f0f9 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:27.612673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:27.615581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:27.615581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:27.615581 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:27.621853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:27.623549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6de376-ae92-4152-a209-39ab6a340155 service swift] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:27.632027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1304/2625] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 05:37:27.740300 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244dc46c80544cadab0ab1b2846e14db', 'request_id': 'req-81eacba5-d9f5-4268-a6ae-801604283051'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.847005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244dc46c80544cadab0ab1b2846e14db', 'request_id': 'req-81eacba5-d9f5-4268-a6ae-801604283051'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.854263 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244dc46c80544cadab0ab1b2846e14db', 'request_id': 'req-81eacba5-d9f5-4268-a6ae-801604283051'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.865807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] The token cache is being invalidate because project 244dc46c80544cadab0ab1b2846e14db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:27.866098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.866782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-81eacba5-d9f5-4268-a6ae-801604283051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:27.868312 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1322/2626] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:27 2022] DELETE /identity/v3/projects/244dc46c80544cadab0ab1b2846e14db => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:27.873265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:27.873656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:27.873933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:27.923123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:27.924328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:27.985662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1305/2627] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:27.993755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:27.994041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:27.994369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.014314 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:28.015812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.024584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1323/2628] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:28.028813 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.038789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d0463-7275-427a-8deb-ffe0d31594a3 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.043624 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.044228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.044514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.044789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.045484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:28.045786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.047415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.063046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1306/2629] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:37:28.064460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.073963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dabee-c222-49d1-8b94-1fbc20b96b84 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.082925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.082925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.082925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.082925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.083462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.083462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062825da-f86d-4a3c-8f6e-a27370f8aea1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.201061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1324/2630] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:28.202133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.213039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c2bff-d536-4112-8635-212f7d95e8be admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.273851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.274520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.274806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.275082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.275721 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.277771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.309400 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.313055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.313055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.330923 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] MFA Rules not processed for user `d3b59fdbefe24db892bb40b4b23fe78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:28.332165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.371435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:28.377048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1307/2631] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:28.378156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.383453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:28.388800 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d9c7-6ab8-4bad-960b-fac51c3bbefe admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.389614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1325/2632] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:37:28.393598 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.394121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.394417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.394719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.395371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.396982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.403329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.404762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.408431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1308/2633] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:28.412493 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.422287 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66200ae-b649-4906-91e6-7d6b94e16c1d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.427504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.429397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.429397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.429397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.429397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.431279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.441702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1309/2634] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:37:28.445864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.456586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c14b9-8f1d-4d15-a1b5-3b6741d6f459 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.461151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.461778 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.462057 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.462371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] PATH_INFO: `/v3/projects/d5fba87c7ee942d9baa32b43fd851917/users/ebc5915501ce49c89e9e14c9a1a7d172/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.474889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b84a951-d5c8-44e9-a08c-5d60148c3c84 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.477246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5fba87c7ee942d9baa32b43fd851917, user_id=ebc5915501ce49c89e9e14c9a1a7d172, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.480059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.480557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01a4e7f4d1e54b17938c3514428a34b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.483055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.483055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.483055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.490509 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.492114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863e12d-c289-455a-8234-388e28817a1b service swift] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.500054 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1326/2635] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 05:37:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 05:37:28.568718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1310/2636] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:28 2022] PUT /identity/v3/projects/d5fba87c7ee942d9baa32b43fd851917/users/ebc5915501ce49c89e9e14c9a1a7d172/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:28.580944 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.581227 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.581503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.607342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.609079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.621866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] MFA Rules not processed for user `ebc5915501ce49c89e9e14c9a1a7d172`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:28.623029 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.692265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1327/2637] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:28.696992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.707560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678b84f-c375-48cc-ba75-773776a24958 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.713700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.714321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.714685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.714966 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.717032 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:28.732255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.733854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.744538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1311/2638] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:28.771119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23da26-16ee-4121-803a-0c7325a1995d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.775823 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.776446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.776767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.777041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.777686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.779682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:28.795889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:28.862491 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1328/2639] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:28.864910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:28.876873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152605ee-a2fc-41d2-a802-e277c012d549 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:28.979059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:28.979823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:28.980168 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:28.980508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:28.981217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:28.983444 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.053129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1312/2640] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:28 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:29.058358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:29.070343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724878-851e-400b-b3ef-92146e01491b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:29.074965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:29.077700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.077700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.077700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.077700 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:29.078339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.090464 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1329/2641] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:29.118061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:29.129067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae85f81-8f1c-47e7-b553-7abc2d6e8274 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:29.144323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:29.144323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.144323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.144323 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] PATH_INFO: `/v3/projects/65209a27fe4c4812972ee64e8da63585/users/6488a623a150488dbc58f30999ddfda0/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.157622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:29.160882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65209a27fe4c4812972ee64e8da63585, user_id=6488a623a150488dbc58f30999ddfda0, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:29.163549 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.227578 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1313/2642] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:29 2022] PUT /identity/v3/projects/65209a27fe4c4812972ee64e8da63585/users/6488a623a150488dbc58f30999ddfda0/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:29.231260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:29.242143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82c9b6-071d-4cf7-93cd-215e629c5a72 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:29.314195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:29.315195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.315526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.315963 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.316900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:29.318746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30295d7f-27ec-46e0-8d45-168e1f99dc25 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.331448 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1330/2643] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:29.335634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:29.347890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbcf2a-a5c5-4826-81bb-eca1bc609736 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:29.353378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:29.354181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.354550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.354890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] PATH_INFO: `/v3/projects/65209a27fe4c4812972ee64e8da63585/users/6488a623a150488dbc58f30999ddfda0/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.356983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65209a27fe4c4812972ee64e8da63585, user_id=6488a623a150488dbc58f30999ddfda0, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:29.360964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.375246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1314/2644] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:29 2022] PUT /identity/v3/projects/65209a27fe4c4812972ee64e8da63585/users/6488a623a150488dbc58f30999ddfda0/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:29.386173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.386274 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.386582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.423415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] MFA Rules not processed for user `6488a623a150488dbc58f30999ddfda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:29.425254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:29.498334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1331/2645] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:29.509984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:29.511810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:29.572438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba30c81-9268-413a-b2ad-2209b23d36c2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:29.576118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:29.576716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:29.576990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:29.577276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:29.636344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:29.637818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7db609-0a9a-47b9-8d84-3c762daf9edf service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:29.647696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1315/2646] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:31.213887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:31.215412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:31.278753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8cbe9-d6e1-438f-bad0-0c7fd7db95a2 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:31.283861 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:31.284480 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:31.284753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:31.285025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] PATH_INFO: `/v3/users/6232623dd58e40008a756b6904e7a612` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:31.286130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6232623dd58e40008a756b6904e7a612)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:31.288072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:31.290289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:31.455592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6232623dd58e40008a756b6904e7a612'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:31.467242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6232623dd58e40008a756b6904e7a612'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:31.474593 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6232623dd58e40008a756b6904e7a612'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:31.475254 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6232623dd58e40008a756b6904e7a612'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:31.475854 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6232623dd58e40008a756b6904e7a612'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:31.554838 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1332/2647] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:31 2022] DELETE /identity/v3/users/6232623dd58e40008a756b6904e7a612 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:31.559455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:31.559944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:31.560047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:31.580392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:31.581608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:31.646152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1316/2648] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:31.656083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:31.657529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:31.717013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68845bb-e972-4276-ab5d-4dc109f03b6b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:31.723987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:31.724295 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:31.724576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:31.724930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:31.737033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:31.738618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb38d8-6c83-4cd5-9bf3-264e8b7ace65 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:31.749242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1333/2649] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:31.991042 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:31.993012 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:32.006485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3fa55-8fb1-4371-ac8b-d55c2f73822a None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.011475 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.012119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.012438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.012711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] PATH_INFO: `/v3/projects/6d8b94c6caa34c178d85187b2147a437` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.013992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8b94c6caa34c178d85187b2147a437)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.015981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.122741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:32.138083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8b94c6caa34c178d85187b2147a437', 'request_id': 'req-cbbb68aa-7949-4715-ab0f-0b86d0919055'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:32.143517 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8b94c6caa34c178d85187b2147a437', 'request_id': 'req-cbbb68aa-7949-4715-ab0f-0b86d0919055'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:32.199991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8b94c6caa34c178d85187b2147a437', 'request_id': 'req-cbbb68aa-7949-4715-ab0f-0b86d0919055'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:32.211339 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] The token cache is being invalidate because project 6d8b94c6caa34c178d85187b2147a437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:32.211626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:32.212249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-cbbb68aa-7949-4715-ab0f-0b86d0919055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:32.213714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1317/2650] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:31 2022] DELETE /identity/v3/projects/6d8b94c6caa34c178d85187b2147a437 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:32.218926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.224592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.224592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.241832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:32.241832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:32.298151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1334/2651] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:32.306507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.306789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.307062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.327126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:32.328336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:32.336267 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1318/2652] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:32.340288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.350292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd8d07-742b-44f5-afcc-be935e22ba03 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.354871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.355463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.355745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.356018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.356661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:32.356964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.358584 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.370354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1335/2653] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:37:32.374382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.383567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710a2b7-f186-4e44-bdb3-fcb1e655a4f1 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.388159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.388742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.389018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.389315 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.389945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.391878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.488072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1319/2654] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:32 2022] POST /identity/v3/projects => generated 397 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:32.496900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.502696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a38e7-b5d1-4da2-bd3c-aa80fc25a68f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.510241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:32.558796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.559399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.559679 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.559950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.560570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.562568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.687120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1336/2655] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:32 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:32.691316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.701590 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbf965-b229-4e3c-be10-f13ee0c55eed admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.706701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.707294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.707570 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.707840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.708481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.710082 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.720117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1320/2656] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:32.724539 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.734977 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd59b56-a300-4406-97cd-3ecca7b230bd admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.739759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.740337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.740610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.740880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.741516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.743140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321985a-24a3-4bfd-9a08-4ae6bf272568 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.753540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1337/2657] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:32.757413 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:32.767616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529b533-e216-4c0f-8ddd-08e441e14fed admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:32.772091 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:32.772728 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.773007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.773274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] PATH_INFO: `/v3/projects/d8c2fb98452142d19ece30c7b386c96d/users/52d12207938c4fd98dca287f44d067df/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.789000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c2fb98452142d19ece30c7b386c96d, user_id=52d12207938c4fd98dca287f44d067df, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:32.791631 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:32.890347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1321/2658] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:32 2022] PUT /identity/v3/projects/d8c2fb98452142d19ece30c7b386c96d/users/52d12207938c4fd98dca287f44d067df/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:32.897947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:32.898231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:32.898555 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:32.927301 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] MFA Rules not processed for user `52d12207938c4fd98dca287f44d067df`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:32.928639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:32.992079 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1338/2659] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:33.003255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:33.005006 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:33.060480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6c41f-df95-453b-9c39-7826f68b0588 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:33.064848 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:33.065480 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:33.065758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:33.066029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:33.074636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:33.081249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:33.085559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1322/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:33 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:35.981579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:35.983028 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:35.995701 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fdd7f-b241-4513-8bd7-f3df5bae5235 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:35.999327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:35.999926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:36.000205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:36.000476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:36.002789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:36.066632 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:36.068125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fdc67-ec18-47ca-8310-5bb7d612ac62 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:36.077647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1339/2661] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:43.081471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:44.096302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:44.157901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8866e18-864d-48b2-b1d9-1067f8127d7f service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:44.162821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:44.163436 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:44.163713 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:44.163983 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] PATH_INFO: `/v3/users/52d12207938c4fd98dca287f44d067df` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:44.165085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d12207938c4fd98dca287f44d067df)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:44.167044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:44.275415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d12207938c4fd98dca287f44d067df'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:44.282318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d12207938c4fd98dca287f44d067df'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:44.331693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d12207938c4fd98dca287f44d067df'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:44.332814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d12207938c4fd98dca287f44d067df'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:44.345645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d12207938c4fd98dca287f44d067df'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:44.352497 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1323/2662] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:37:44 2022] DELETE /identity/v3/users/52d12207938c4fd98dca287f44d067df => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:44.357167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:44.357669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:44.357787 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:44.377319 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:44.378536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:44.386476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:44.443135 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1340/2663] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:44.452974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:44.454641 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:44.533529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054b8b-877e-4ed1-93ef-646b41e870b7 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:44.537763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:44.538379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:44.538667 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:44.538935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:44.547873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:44.549312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06363f82-bb8e-4cee-b969-819362694d57 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:44.558922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1324/2664] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:44.955195 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:44.956610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:44.969887 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745a237-67c1-463c-8a38-18336f59c5d7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:44.974714 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:44.975356 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:44.975771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:44.975930 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] PATH_INFO: `/v3/projects/d8c2fb98452142d19ece30c7b386c96d` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:44.977068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c2fb98452142d19ece30c7b386c96d)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:44.979036 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.108577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c2fb98452142d19ece30c7b386c96d', 'request_id': 'req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:45.182273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c2fb98452142d19ece30c7b386c96d', 'request_id': 'req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:45.187962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c2fb98452142d19ece30c7b386c96d', 'request_id': 'req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:45.198835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] The token cache is being invalidate because project d8c2fb98452142d19ece30c7b386c96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:37:45.199114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:45.199725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-92cce2f3-9dee-49db-b6b7-fbf1ec02ee7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:37:45.201180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1341/2665] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:37:44 2022] DELETE /identity/v3/projects/d8c2fb98452142d19ece30c7b386c96d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:45.206087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.206591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.206701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.214634 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:45.234391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:45.234391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:45.287747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1325/2666] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:45.350085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.350348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.350660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.369970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:45.371248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:45.379022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1342/2667] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:45.383256 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.393284 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41efd2-4a51-4924-a7a5-f5e7959f361d None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.398583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.399234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.399563 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.399862 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.400663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:37:45.400988 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.402730 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.415213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1326/2668] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:37:45.419554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.428837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df81610-d601-4f39-b1b6-53df7b1263c7 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.433379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.433990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.434260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.434559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.435178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.437064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.515553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1343/2669] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:45 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:45.520263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.530577 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45e305-c2ac-4a49-bc48-ecfcce1864f6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.587974 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.588580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.588860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.589153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.589780 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.591752 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.680416 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1327/2670] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:45 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:45.684622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.695425 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ca54-cd24-4919-ab72-c80154d0b881 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.700041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.700626 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.700900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.701168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.701802 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.703399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.713179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1344/2671] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:45.716917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.727765 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007a1be-16e3-47b3-8d7e-e5120a09fd9a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.732467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.733059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.733336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.733608 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.734247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.735875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d59b9-ae43-44d2-bf6e-10ecd4190e87 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.746872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1328/2672] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:45.750824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.760505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3933cbe-711c-4318-9010-c60327411860 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:45.765034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:45.765720 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.765945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.766217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] PATH_INFO: `/v3/projects/42a5c327025a4c04bde73c77acc4a452/users/654c715e5d32487d863a65bbc77ff561/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.781248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a5c327025a4c04bde73c77acc4a452, user_id=654c715e5d32487d863a65bbc77ff561, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:45.783926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:45.846023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1345/2673] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:45 2022] PUT /identity/v3/projects/42a5c327025a4c04bde73c77acc4a452/users/654c715e5d32487d863a65bbc77ff561/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:45.851922 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251ba3-431e-4b33-86d4-a45258316ffa None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:45.852212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251ba3-431e-4b33-86d4-a45258316ffa None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:45.852488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251ba3-431e-4b33-86d4-a45258316ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:45.871246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-83251ba3-431e-4b33-86d4-a45258316ffa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:45.875308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-83251ba3-431e-4b33-86d4-a45258316ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:45.933341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1329/2674] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:45.942818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:45.944511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:45.979716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:46.000585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c7d33-4dd1-4056-a35d-1d704e4a6511 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:46.004345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:46.005142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:46.005468 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:46.005939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:46.015307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:46.016771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:46.026574 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1346/2675] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:37:51.221183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.221696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.221810 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.242806 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:51.251932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] MFA Rules not processed for user `654c715e5d32487d863a65bbc77ff561`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:51.253224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:51.328192 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1330/2676] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:51.333434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.343676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be759238-1210-48b6-baaa-383fc2879feb service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.349243 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.349987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.350108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.350596 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.351257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.353173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.487671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1347/2677] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:37:51 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:37:51.487839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.498154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88569b2-a1c7-491f-9442-cadbde98be2e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.561176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.561808 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.562090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.562363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.563058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.565100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.655738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1331/2678] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:37:51 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:37:51.657112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.672591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd72b4-6196-451d-8914-31494b664013 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.672591 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.673280 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.673558 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.673829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.674516 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.676806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.687694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1348/2679] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:51.692372 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.702909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138c278-ddef-4377-a6b5-adedee39f2eb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.708180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.708839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.709121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.709396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] PATH_INFO: `/v3/projects/f24d64ac4f064540a413be87910f83c2/users/2a11dfd4735b4c378340e122e01694b0/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.727342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24d64ac4f064540a413be87910f83c2, user_id=2a11dfd4735b4c378340e122e01694b0, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.730250 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.829013 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1332/2680] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:51 2022] PUT /identity/v3/projects/f24d64ac4f064540a413be87910f83c2/users/2a11dfd4735b4c378340e122e01694b0/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:37:51.832982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.842964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e02aa7-48d2-4a90-b0d7-288c64529acc admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.899458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.900067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.900354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.900629 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.901272 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.902932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc8f09-e274-40e2-8b1b-c6ed30faa443 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.912501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1349/2681] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:37:51.916705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:51.927490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01529f1a-800d-45ae-8a5e-587836995104 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:51.932628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:51.933328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:51.933534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:51.933805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] PATH_INFO: `/v3/projects/f24d64ac4f064540a413be87910f83c2/users/2a11dfd4735b4c378340e122e01694b0/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:51.935757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f24d64ac4f064540a413be87910f83c2, user_id=2a11dfd4735b4c378340e122e01694b0, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:51.938392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:51.950295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1333/2682] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:37:51 2022] PUT /identity/v3/projects/f24d64ac4f064540a413be87910f83c2/users/2a11dfd4735b4c378340e122e01694b0/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:37:57.369805 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c3d7-8dbb-476e-96af-4abdc21c6b94 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:57.370725 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c3d7-8dbb-476e-96af-4abdc21c6b94 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:57.371000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460c3d7-8dbb-476e-96af-4abdc21c6b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:57.401510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-2460c3d7-8dbb-476e-96af-4abdc21c6b94 None None] MFA Rules not processed for user `2a11dfd4735b4c378340e122e01694b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:57.402797 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2460c3d7-8dbb-476e-96af-4abdc21c6b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:57.475101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1350/2683] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:37:57.490892 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:37:57.492625 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:57.556230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:37:57.611007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e79910-831b-48ae-b36b-a7a0f03cdf72 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:37:57.618212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:37:57.618872 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:57.619154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:57.619432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:57.680541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:37:57.682145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:37:57.691523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1334/2684] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:37:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:37:59.892734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b2f1-ed70-40db-933a-801d687d0725 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:37:59.893278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b2f1-ed70-40db-933a-801d687d0725 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:37:59.893404 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100b2f1-ed70-40db-933a-801d687d0725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:37:59.913460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5100b2f1-ed70-40db-933a-801d687d0725 None None] MFA Rules not processed for user `654c715e5d32487d863a65bbc77ff561`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:37:59.916193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5100b2f1-ed70-40db-933a-801d687d0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:37:59.921868 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1351/2685] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:37:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:05.424087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:05.425997 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:05.491453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbd5d6-5a18-45a4-832a-07654d64eecc service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:05.496126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:05.496866 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:05.497146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:05.497458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] PATH_INFO: `/v3/users/ebc5915501ce49c89e9e14c9a1a7d172` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:05.498610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc5915501ce49c89e9e14c9a1a7d172)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:05.501059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:05.677724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc5915501ce49c89e9e14c9a1a7d172'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:05.684674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc5915501ce49c89e9e14c9a1a7d172'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:05.781771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc5915501ce49c89e9e14c9a1a7d172'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:05.782434 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc5915501ce49c89e9e14c9a1a7d172'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:05.789055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:05.797097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc5915501ce49c89e9e14c9a1a7d172'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:05.804199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1335/2686] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:05 2022] DELETE /identity/v3/users/ebc5915501ce49c89e9e14c9a1a7d172 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:05.808669 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:05.809226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:05.809226 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:05.829014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:05.830245 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:05.842000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:05.890262 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1352/2687] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:05.900296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:05.901689 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:05.959129 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271df22-eb37-4eba-aea7-0ecd17e26b84 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:05.963023 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:05.963711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:05.963985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:05.964253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:05.975762 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:05.977228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:05.986981 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1336/2688] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:38:06.386331 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:06.387814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:06.399818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d89ed-e86e-46c2-9ed2-b61177a6dabe None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:06.404456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:06.405090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:06.405799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:06.405799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] PATH_INFO: `/v3/projects/d5fba87c7ee942d9baa32b43fd851917` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:06.406834 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5fba87c7ee942d9baa32b43fd851917)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:06.408800 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:06.645110 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5fba87c7ee942d9baa32b43fd851917', 'request_id': 'req-acb00576-c227-4485-8872-8d2968244755'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.713307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5fba87c7ee942d9baa32b43fd851917', 'request_id': 'req-acb00576-c227-4485-8872-8d2968244755'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.719595 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5fba87c7ee942d9baa32b43fd851917', 'request_id': 'req-acb00576-c227-4485-8872-8d2968244755'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.731503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] The token cache is being invalidate because project d5fba87c7ee942d9baa32b43fd851917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:06.731789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.732415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.733888 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1353/2689] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:06 2022] DELETE /identity/v3/projects/d5fba87c7ee942d9baa32b43fd851917 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:06.738771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:06.749635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713eacd-0f5a-437e-8856-01aa23c5ddd8 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:06.807716 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:06.808381 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:06.808664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:06.808937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] PATH_INFO: `/v3/users/6488a623a150488dbc58f30999ddfda0` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:06.810059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6488a623a150488dbc58f30999ddfda0)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:06.812117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:06.979526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6488a623a150488dbc58f30999ddfda0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:06.988432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6488a623a150488dbc58f30999ddfda0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:07.061918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6488a623a150488dbc58f30999ddfda0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:07.062692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6488a623a150488dbc58f30999ddfda0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:07.075622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b4fafddc-9501-4cad-b2aa-0e3626519d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6488a623a150488dbc58f30999ddfda0'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:07.082391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1337/2690] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:06 2022] DELETE /identity/v3/users/6488a623a150488dbc58f30999ddfda0 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:07.755616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:07.757225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:07.770500 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb00576-c227-4485-8872-8d2968244755 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:07.828770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:07.829318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:07.829590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:07.829858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] PATH_INFO: `/v3/projects/65209a27fe4c4812972ee64e8da63585` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:07.831533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] RBAC: Authorizing `identity:delete_project(project_id=65209a27fe4c4812972ee64e8da63585)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:07.833467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:07.950617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:07.954397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65209a27fe4c4812972ee64e8da63585', 'request_id': 'req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:08.050208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65209a27fe4c4812972ee64e8da63585', 'request_id': 'req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:08.056703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65209a27fe4c4812972ee64e8da63585', 'request_id': 'req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:08.068609 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] The token cache is being invalidate because project 65209a27fe4c4812972ee64e8da63585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:08.068897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:08.069524 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-d60177ee-c396-4dc5-bd6c-25eb1ebea8db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:08.071026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1354/2691] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:07 2022] DELETE /identity/v3/projects/65209a27fe4c4812972ee64e8da63585 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:08.076005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.076518 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.076632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.098027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:08.099377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:08.104159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:08.164479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1338/2692] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:38:08.172180 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.172461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.172734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.193002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:08.194264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:08.202663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1355/2693] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:08.209531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.217603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774e77e-bddc-47c1-ab1b-65d241f9da49 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.222883 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.223510 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.223791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.224064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.224715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:38:08.225019 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.226710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.238719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1339/2694] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 05:38:08.243633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.253782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c240a-7aa7-4bc1-8a50-c51a19019eda None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.258474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.259083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.259375 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.259641 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.260270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.262169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.331894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1356/2695] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/projects => generated 399 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:38:08.335930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.347079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a0af1-6dab-4acf-998e-31816bce7baa admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.408687 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.409318 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.409599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.409873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.410543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.412821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.513066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1340/2696] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:38:08.517361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.529051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76970a5a-485a-432b-a7d0-53743b089d7d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.533607 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.534282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.534592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.534862 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.535501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.537096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.547335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1357/2697] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:08.551968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.562181 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22606c9d-b33d-41c6-888d-748f031b5c2c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.567514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.568133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.568411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.568686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.569317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.570942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0809d-db5d-4f70-b86e-9e4001c135a1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.581823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1341/2698] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:08.586008 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.596134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc60eb-d00e-478d-892b-c0245cced57e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.600808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.601443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.601718 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.601987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] PATH_INFO: `/v3/projects/dafb5b5fa36b40769394614293ac1aff/users/06941f4db2594361b039fe0ecd314000/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.617189 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafb5b5fa36b40769394614293ac1aff, user_id=06941f4db2594361b039fe0ecd314000, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.619835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.690347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1358/2699] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:08 2022] PUT /identity/v3/projects/dafb5b5fa36b40769394614293ac1aff/users/06941f4db2594361b039fe0ecd314000/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:08.697559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.697842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.698143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.725920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] MFA Rules not processed for user `06941f4db2594361b039fe0ecd314000`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:08.727162 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:08.797457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1342/2700] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:08.803281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.868568 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88e18-ede1-404c-85ff-703c1ee42064 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:08.873338 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:08.874438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:08.874438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:08.874660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:08.875208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:08.877153 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:08.966675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1359/2701] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/projects => generated 401 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:08.970797 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:08.978084 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:08.982240 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba754f-7415-4ef6-95b4-f386dd9fc5e6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.040128 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.040739 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.041026 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.041296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.041936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.043936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.119686 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1343/2702] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:08 2022] POST /identity/v3/users => generated 307 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:09.123501 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.134273 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291bcc-2a44-4f88-8397-3c83d427be33 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.138956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.139557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.139832 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.140101 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.140745 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.142369 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.154095 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1360/2703] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:09.159682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.170697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece21c2-411e-403f-bab6-0b88fc49f286 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.175836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.176538 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.176818 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.177338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] PATH_INFO: `/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930/users/162c1e502e064d5aa36a93fca07ea65d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.192794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5adce4b2ecce490a9eb1cf5a1f485930, user_id=162c1e502e064d5aa36a93fca07ea65d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.195504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.262754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1344/2704] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:09 2022] PUT /identity/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930/users/162c1e502e064d5aa36a93fca07ea65d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:38:09.266893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.278043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306d552-7855-4227-bdde-4a401fed8df5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.318399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:09.334692 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.335305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.335589 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.335857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.336751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.338269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d88f0-92d8-46d9-9efc-4963942d1794 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.349736 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1361/2705] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:09.353942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.364247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ec91-387e-40d6-ab9a-77cea0544bb3 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.368837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.369512 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.369841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.370068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] PATH_INFO: `/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930/users/162c1e502e064d5aa36a93fca07ea65d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.372009 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5adce4b2ecce490a9eb1cf5a1f485930, user_id=162c1e502e064d5aa36a93fca07ea65d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.374692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.385028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1345/2706] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:09 2022] PUT /identity/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930/users/162c1e502e064d5aa36a93fca07ea65d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:09.396708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.397030 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.397307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.424881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] MFA Rules not processed for user `162c1e502e064d5aa36a93fca07ea65d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:09.426146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:09.503223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1362/2707] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:09 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:09.514292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.516031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:09.596829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2074e-e52e-4a5c-b41c-28ecc2f209bf admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.600509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.601108 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.602370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.602676 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.612153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.613693 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.623973 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1346/2708] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:09.847905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:09.857859 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833858e-80ac-44da-a754-c3fb70e7d974 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:09.861457 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:09.862060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:09.862339 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:09.862644 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:09.920445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:09.921796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec521c7b-cd3c-4ebd-88b9-4e9b8d4c831d service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:09.931419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1363/2709] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:09 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:14.800919 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:14.843302 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:14.862107 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8614-4be2-48fa-8c8f-ad3bfde3b4df service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:14.866910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:14.867679 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:14.868338 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:14.868681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] PATH_INFO: `/v3/users/06941f4db2594361b039fe0ecd314000` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:14.869801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=06941f4db2594361b039fe0ecd314000)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:14.871826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:15.017887 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06941f4db2594361b039fe0ecd314000'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.024700 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06941f4db2594361b039fe0ecd314000'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.072607 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06941f4db2594361b039fe0ecd314000'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.073228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06941f4db2594361b039fe0ecd314000'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.085048 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06941f4db2594361b039fe0ecd314000'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.090942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1347/2710] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:14 2022] DELETE /identity/v3/users/06941f4db2594361b039fe0ecd314000 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:15.095439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:15.095954 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:15.096072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:15.114904 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:15.116093 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:15.172034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1364/2711] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:15.182594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:15.184011 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:15.240151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e036bc-c38a-450d-a369-fa78e756bcaf admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:15.243863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:15.244459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:15.244737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:15.245015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:15.253943 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:15.255990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:15.265305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1348/2712] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:15.697324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:15.698846 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:15.707134 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=100 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:15.713128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3165560-61b1-43b3-bf10-b609b01d68fb None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:15.717820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:15.718572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:15.718852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:15.719118 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] PATH_INFO: `/v3/projects/dafb5b5fa36b40769394614293ac1aff` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:15.720234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafb5b5fa36b40769394614293ac1aff)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:15.722130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:15.862152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafb5b5fa36b40769394614293ac1aff', 'request_id': 'req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.929316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafb5b5fa36b40769394614293ac1aff', 'request_id': 'req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.935055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafb5b5fa36b40769394614293ac1aff', 'request_id': 'req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.946707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] The token cache is being invalidate because project dafb5b5fa36b40769394614293ac1aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:15.946993 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.947723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:15.949207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1365/2713] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:15 2022] DELETE /identity/v3/projects/dafb5b5fa36b40769394614293ac1aff => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:15.953212 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:15.962836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b1818-d574-4b32-9304-8b1ce3859688 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:16.019000 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:16.019791 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:16.020077 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:16.020351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] PATH_INFO: `/v3/users/162c1e502e064d5aa36a93fca07ea65d` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:16.021652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=162c1e502e064d5aa36a93fca07ea65d)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:16.023642 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:16.025971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:16.303875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162c1e502e064d5aa36a93fca07ea65d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:16.310847 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162c1e502e064d5aa36a93fca07ea65d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:16.381037 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162c1e502e064d5aa36a93fca07ea65d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:16.381696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162c1e502e064d5aa36a93fca07ea65d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:16.393851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162c1e502e064d5aa36a93fca07ea65d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:16.399662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1349/2714] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:15 2022] DELETE /identity/v3/users/162c1e502e064d5aa36a93fca07ea65d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:16.880513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:16.883567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:16.897922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea24c1-5f38-4002-b262-4c5c5d4d87c2 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:16.923673 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:16.925170 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:16.957521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:16.958293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:16.958603 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:16.958932 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] PATH_INFO: `/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:16.960590 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5adce4b2ecce490a9eb1cf5a1f485930)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:16.962671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:16.987356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff54a9-2d33-4867-af2d-c978432c8692 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:16.991005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:16.991719 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:16.991992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:16.992262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:16.993967 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:17.003677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.005130 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c64733-afea-4a11-a61d-84b3e3304850 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.013863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1350/2715] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:38:17.044417 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5adce4b2ecce490a9eb1cf5a1f485930', 'request_id': 'req-ebcd29ec-bb4a-462b-bbb5-680384888ea0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:17.089622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5adce4b2ecce490a9eb1cf5a1f485930', 'request_id': 'req-ebcd29ec-bb4a-462b-bbb5-680384888ea0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:17.095640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5adce4b2ecce490a9eb1cf5a1f485930', 'request_id': 'req-ebcd29ec-bb4a-462b-bbb5-680384888ea0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:17.107398 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] The token cache is being invalidate because project 5adce4b2ecce490a9eb1cf5a1f485930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:17.107690 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:17.108341 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-ebcd29ec-bb4a-462b-bbb5-680384888ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:17.109809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1366/2716] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:16 2022] DELETE /identity/v3/projects/5adce4b2ecce490a9eb1cf5a1f485930 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:17.114965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.115270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.115543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.136523 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:17.137748 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:17.199459 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1351/2717] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:38:17.207385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.207674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.207972 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.228392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:17.229601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:17.237649 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1367/2718] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:17.242160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.252173 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313c1d9-208c-4510-9a4e-349f7a3ae629 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.261329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.261329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.261329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.261329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.261767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:38:17.261767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.261767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.273711 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1352/2719] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:38:17.277843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.288166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4d09e-16d2-440f-a5f5-ba864af0e65f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.292829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.293437 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.293696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.293959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.294819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.296461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.307391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:17.411129 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1368/2720] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/projects => generated 381 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:17.416231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.426054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fabc635-5a8b-443f-8e56-f5acd60b9aee admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.434472 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:17.482715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.483324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.483636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.483932 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.484564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.486551 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.587568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1353/2721] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:17.591788 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.602092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150dbaf-2ab4-49e4-b7ea-7941637b897e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.610585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.610585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.610585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.610585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.610986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.610986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.624233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1369/2722] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:17.625727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.637672 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839c590-5c79-46fb-acd4-a40eb6fed002 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.644701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.645332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.645630 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.645901 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.646601 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.648841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae4c5a-18ca-40a9-803e-f38eabd83989 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.659515 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1354/2723] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:17.663716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.673634 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d018b54-79fd-48a9-b842-fce84d2a448f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.678456 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.679126 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.679403 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.679670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] PATH_INFO: `/v3/projects/b2f8df6192de4188b37ac120ed452f1a/users/167e6785e22645d7a634f4818cb26aaa/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.694246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f8df6192de4188b37ac120ed452f1a, user_id=167e6785e22645d7a634f4818cb26aaa, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.697445 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:17.776732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1370/2724] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:17 2022] PUT /identity/v3/projects/b2f8df6192de4188b37ac120ed452f1a/users/167e6785e22645d7a634f4818cb26aaa/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:38:17.787640 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.788017 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.788414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.820769 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] MFA Rules not processed for user `167e6785e22645d7a634f4818cb26aaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:17.821987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:17.889392 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1355/2725] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:17.893812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:17.955645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18d2cd-e1f2-4f3c-a74b-fe4142894d31 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:17.960393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:17.960994 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:17.961275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:17.961550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:17.962305 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:17.964494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.113776 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1371/2726] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:17 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:18.118415 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.128918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfcf1f-0092-4159-aa15-0e8f2e944072 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.187163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.187777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.188058 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.188324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.188975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.191101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.324406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1356/2727] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:18 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:38:18.328277 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.338387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420adb07-4aec-4c9f-a72c-061f8b73f249 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.342942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.345482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.345482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.345482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.345482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.346504 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.358863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1372/2728] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:18.363311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.374100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa214-8a33-4f7c-b262-c9fce0427c6d admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.378604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.379247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.379526 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.379798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] PATH_INFO: `/v3/projects/a7bb2b0937f64af2a253e331fbfbe923/users/76b331c36c3a4a398a56b55aa58c710a/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.382977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:18.394613 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bb2b0937f64af2a253e331fbfbe923, user_id=76b331c36c3a4a398a56b55aa58c710a, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.397421 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.496024 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1357/2729] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:18 2022] PUT /identity/v3/projects/a7bb2b0937f64af2a253e331fbfbe923/users/76b331c36c3a4a398a56b55aa58c710a/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:18.500051 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.509872 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7fec0-d921-4c81-af94-599d5ee90230 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.566462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.567067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.567347 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.567653 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.568287 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.569900 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c833f8-3f76-4cd9-aeb4-9b2c7977295d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.579533 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1373/2730] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:18.583707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.593992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9414d-5535-4f82-b35d-b48fa9ad3e13 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.598591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.599230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.599509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.599776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] PATH_INFO: `/v3/projects/a7bb2b0937f64af2a253e331fbfbe923/users/76b331c36c3a4a398a56b55aa58c710a/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.602166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bb2b0937f64af2a253e331fbfbe923, user_id=76b331c36c3a4a398a56b55aa58c710a, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.604829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.614860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1358/2731] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:18 2022] PUT /identity/v3/projects/a7bb2b0937f64af2a253e331fbfbe923/users/76b331c36c3a4a398a56b55aa58c710a/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:18.682767 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.683050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.683323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.712145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] MFA Rules not processed for user `76b331c36c3a4a398a56b55aa58c710a`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:18.713377 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:18.722698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:18.781260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1374/2732] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:18 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:18.793889 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:18.795636 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:18.858303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39c5b-35e5-47b2-971d-b24beb1fe1f0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:18.862232 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:18.862945 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:18.863319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:18.863662 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:18.873610 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:18.875247 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:18.885153 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1359/2733] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:19.024340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:19.034237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e99d74-e13c-4b64-afe7-0e6ac221cb56 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:19.037953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:19.039131 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:19.042010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:19.042010 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:19.095012 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:19.096485 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05cf6-ac9a-4d36-b721-ef48f2f020ca service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:19.106179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1375/2734] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:19.589531 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:19.670925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61036605-ac43-4103-8023-6cbd3977419f service cinder] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:19.670925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:19.670925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:19.671448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:19.671726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] PATH_INFO: `/v3/users/167e6785e22645d7a634f4818cb26aaa` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:19.672865 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167e6785e22645d7a634f4818cb26aaa)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:19.674842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:19.851674 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167e6785e22645d7a634f4818cb26aaa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:19.859072 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167e6785e22645d7a634f4818cb26aaa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:19.929718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167e6785e22645d7a634f4818cb26aaa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:19.930356 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167e6785e22645d7a634f4818cb26aaa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:19.945220 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:19.947745 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167e6785e22645d7a634f4818cb26aaa'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:19.957666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1360/2735] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:19 2022] DELETE /identity/v3/users/167e6785e22645d7a634f4818cb26aaa => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:19.959535 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:19.959820 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:19.960664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:19.983042 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:19.983897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:20.052753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1376/2736] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:20.062241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:20.063705 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:20.120948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5982d-41c0-4b72-aab7-397947ca8848 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:20.124582 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:20.125186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:20.125465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:20.125737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:20.135334 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:20.136858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:20.146628 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1361/2737] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:20.576433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:20.579239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:20.590018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209e4a6-ae14-487c-b27d-2b8f79f46b0b None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:20.594685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:20.595327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:20.595728 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:20.596007 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] PATH_INFO: `/v3/projects/b2f8df6192de4188b37ac120ed452f1a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:20.597244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f8df6192de4188b37ac120ed452f1a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:20.599209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:20.707080 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:20.713120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f8df6192de4188b37ac120ed452f1a', 'request_id': 'req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:20.786340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f8df6192de4188b37ac120ed452f1a', 'request_id': 'req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:20.792561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f8df6192de4188b37ac120ed452f1a', 'request_id': 'req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:20.804370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] The token cache is being invalidate because project b2f8df6192de4188b37ac120ed452f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:20.804659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:20.805289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:20.806809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1377/2738] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:20 2022] DELETE /identity/v3/projects/b2f8df6192de4188b37ac120ed452f1a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:20.810777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:20.820362 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d627cf-9638-4ce9-b6ea-c1244c47ee93 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:20.875592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:20.877462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:20.877462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:20.877462 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] PATH_INFO: `/v3/users/76b331c36c3a4a398a56b55aa58c710a` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:20.877894 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b331c36c3a4a398a56b55aa58c710a)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:20.879875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:21.016647 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b331c36c3a4a398a56b55aa58c710a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.026085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b331c36c3a4a398a56b55aa58c710a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.079871 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b331c36c3a4a398a56b55aa58c710a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.080500 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b331c36c3a4a398a56b55aa58c710a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.096850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-b395bcb4-2ac4-458a-925e-c03fe5a5631b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b331c36c3a4a398a56b55aa58c710a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.104248 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1362/2739] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:20 2022] DELETE /identity/v3/users/76b331c36c3a4a398a56b55aa58c710a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:21.508002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:21.509650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:21.526961 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9be92e-930e-4d4e-b8d3-5ba54b4689f7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:21.584645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:21.585729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:21.586006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:21.586302 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] PATH_INFO: `/v3/projects/a7bb2b0937f64af2a253e331fbfbe923` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:21.587462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bb2b0937f64af2a253e331fbfbe923)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:21.589361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:21.695346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bb2b0937f64af2a253e331fbfbe923', 'request_id': 'req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.746467 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bb2b0937f64af2a253e331fbfbe923', 'request_id': 'req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.752336 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bb2b0937f64af2a253e331fbfbe923', 'request_id': 'req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.763487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] The token cache is being invalidate because project a7bb2b0937f64af2a253e331fbfbe923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:21.763775 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.764399 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-54d2b669-a3c6-455e-bb2c-bf65119ea8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:21.765877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1378/2740] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:21 2022] DELETE /identity/v3/projects/a7bb2b0937f64af2a253e331fbfbe923 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:21.770975 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:21.771270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:21.771552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:21.793902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:21.795427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:21.806380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:21.858691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1363/2741] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:21.866750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:21.870858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:21.870858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:21.886967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:21.888296 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:21.896143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1379/2742] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:21.901158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:21.912102 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18163e0-0ed2-4c06-83e0-cd571c321cbd None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:21.917821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:21.918447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:21.922457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:21.922457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:21.922457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:38:21.922457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:21.922457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:21.933934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1364/2743] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:38:21.938266 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:21.947918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea2c26-3bcc-4504-892f-2f9fe77354ea None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:21.952750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:21.953382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:21.953660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:21.953931 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:21.954637 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:21.956577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.058397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1380/2744] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:21 2022] POST /identity/v3/projects => generated 369 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:22.062899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.074380 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6133ec-095c-4a03-bcd9-d2c1a5612591 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.148639 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.149528 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.149923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.150199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.150925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.153332 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.232021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1365/2745] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:22 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:22.232740 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.248003 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb870-9ab1-4491-8be8-e29dbde57d00 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.254878 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.255490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.255768 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.256035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.256770 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.259430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.276877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1381/2746] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:22.279419 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.292594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b150ed-699e-479f-8bab-91a29dc8f844 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.298123 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.298909 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.299194 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.299468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.300119 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.302304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50442cb5-0049-47cb-add2-5dfc6b72c23f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.315432 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1366/2747] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:22.320169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.330562 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42524405-8ee2-4a5f-87ef-ec1eed2754c9 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.336137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.336793 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.337066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.337335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] PATH_INFO: `/v3/projects/bdedb161d3034f1c87b35512aa3c3376/users/26debdec08fc468fa0eb7c8e090f1e24/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.354294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdedb161d3034f1c87b35512aa3c3376, user_id=26debdec08fc468fa0eb7c8e090f1e24, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.357074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.360643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:22.473320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1382/2748] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:22 2022] PUT /identity/v3/projects/bdedb161d3034f1c87b35512aa3c3376/users/26debdec08fc468fa0eb7c8e090f1e24/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:22.479455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8571e-b246-4b99-b431-eca08fee3bd4 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.479771 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8571e-b246-4b99-b431-eca08fee3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.480047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8571e-b246-4b99-b431-eca08fee3bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.500045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-a0d8571e-b246-4b99-b431-eca08fee3bd4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:22.501269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-a0d8571e-b246-4b99-b431-eca08fee3bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:22.570254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1367/2749] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:22.580619 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:22.582476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:22.643279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2ffea-62cd-48cd-834f-fd3e8a3ead57 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:22.647067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:22.647676 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:22.647951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:22.648321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:22.658125 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:22.659633 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:22.669957 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1383/2750] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:24.111635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.112706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.113137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.149005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] MFA Rules not processed for user `26debdec08fc468fa0eb7c8e090f1e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:24.150255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:24.160508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:24.226703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1368/2751] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:24.231573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.243520 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1579b23-f764-4260-9f3a-87accd8f36e1 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.249232 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.249991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.250588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.251106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.251982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.254640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.401532 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1384/2752] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:24 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:38:24.405727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.420963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a967d25-6a02-4088-b20b-1d40b5c85e4f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.492612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.493304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.493588 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.493864 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.494559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.496519 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.583203 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1369/2753] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:24 2022] POST /identity/v3/users => generated 291 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:24.586953 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.601304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dcb3b-2605-4ef7-8f8c-04477cabd7c1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.607541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.608682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.609150 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.609667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.610554 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.612334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.628687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1385/2754] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:24.637961 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.657283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87644666-5395-403c-8bb0-261c1d9eff13 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.664836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.665567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.665798 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.666106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] PATH_INFO: `/v3/projects/cee5a7d58a774465910841f22a7b7813/users/ccabf78999e642dd920d08158a295d38/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.688126 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee5a7d58a774465910841f22a7b7813, user_id=ccabf78999e642dd920d08158a295d38, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.690681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.770504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1370/2755] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:24 2022] PUT /identity/v3/projects/cee5a7d58a774465910841f22a7b7813/users/ccabf78999e642dd920d08158a295d38/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:24.775685 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.793483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f9350-0cc6-456c-bc03-25962685653c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.879164 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.879874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.880290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.880635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.881367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.883727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc7e01-1719-4e87-8eec-6e56bdeab4ce admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.894910 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1386/2756] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:24.899310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:24.910985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9dafb-5909-41f1-8121-102625b7385f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:24.916180 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:24.916858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:24.917146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:24.917591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] PATH_INFO: `/v3/projects/cee5a7d58a774465910841f22a7b7813/users/ccabf78999e642dd920d08158a295d38/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:24.919591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee5a7d58a774465910841f22a7b7813, user_id=ccabf78999e642dd920d08158a295d38, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:24.923835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:24.934135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1371/2757] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:24 2022] PUT /identity/v3/projects/cee5a7d58a774465910841f22a7b7813/users/ccabf78999e642dd920d08158a295d38/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:38:26.691106 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:26.691612 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:26.691866 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:26.721867 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] MFA Rules not processed for user `ccabf78999e642dd920d08158a295d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:26.723144 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:26.776256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:26.814907 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1387/2758] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:26.823047 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:26.825435 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:26.916501 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f95a-8fc8-41d5-b0a0-e62905121de5 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:26.920265 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:26.920984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:26.921352 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:26.921697 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:26.940346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:26.989572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:26.991101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d7731-be96-46c4-a80b-f5e6842ed70c service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:27.000039 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1372/2759] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:28.764100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:28.767037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:28.783048 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416dc07-62cc-457b-9125-c59ad75ed42c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:28.788064 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:28.789364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:28.789364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:28.789764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:28.800668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:28.802414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:28.813120 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1388/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:29.428402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:29.428911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:29.429054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:29.451018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] MFA Rules not processed for user `9561f323e125411bb83ecce21bb085c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:29.452234 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:29.506070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1373/2761] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:38.808035 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=12 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:38.992002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:38.993512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:39.049706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a68bc-d48d-4b66-87e6-910cf27f9009 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:39.053522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:39.054167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:39.054479 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:39.057809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:39.057809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:39.118569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:39.119997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:39.129217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1389/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:39.499394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=44 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:45.126984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:45.208150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693793b-2a27-4779-ac46-748f3aba8d06 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:45.213245 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:45.213881 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:45.214157 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:45.214443 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] PATH_INFO: `/v3/users/26debdec08fc468fa0eb7c8e090f1e24` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:45.215569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=26debdec08fc468fa0eb7c8e090f1e24)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:45.217525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:45.395546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26debdec08fc468fa0eb7c8e090f1e24'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.403460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26debdec08fc468fa0eb7c8e090f1e24'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.488297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26debdec08fc468fa0eb7c8e090f1e24'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.489100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26debdec08fc468fa0eb7c8e090f1e24'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.502645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26debdec08fc468fa0eb7c8e090f1e24'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.510076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1374/2763] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:45 2022] DELETE /identity/v3/users/26debdec08fc468fa0eb7c8e090f1e24 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:45.768940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:45.776836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cf982-172c-4ae6-bbcd-39b5bbdeb2e8 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:45.834873 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:45.835503 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:45.835783 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:45.836056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] PATH_INFO: `/v3/projects/bdedb161d3034f1c87b35512aa3c3376` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:45.837173 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdedb161d3034f1c87b35512aa3c3376)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:45.839117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:45.949918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdedb161d3034f1c87b35512aa3c3376', 'request_id': 'req-f70e66cf-03c9-4caf-9e71-b64777cdae01'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:45.952449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:46.030755 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdedb161d3034f1c87b35512aa3c3376', 'request_id': 'req-f70e66cf-03c9-4caf-9e71-b64777cdae01'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.036281 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdedb161d3034f1c87b35512aa3c3376', 'request_id': 'req-f70e66cf-03c9-4caf-9e71-b64777cdae01'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.047678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] The token cache is being invalidate because project bdedb161d3034f1c87b35512aa3c3376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:46.047966 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.048597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.050332 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1390/2764] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:45 2022] DELETE /identity/v3/projects/bdedb161d3034f1c87b35512aa3c3376 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:46.054438 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:46.056496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:46.069202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee880f-9794-4dad-9a04-3bc712d406ef admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:46.124402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:46.125028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:46.125304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:46.125575 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] PATH_INFO: `/v3/users/ccabf78999e642dd920d08158a295d38` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:46.126723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccabf78999e642dd920d08158a295d38)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:46.128637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:46.148978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:46.318553 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccabf78999e642dd920d08158a295d38'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.325063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccabf78999e642dd920d08158a295d38'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.408347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccabf78999e642dd920d08158a295d38'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.408960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccabf78999e642dd920d08158a295d38'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.422076 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-2f88e9a1-8c99-4994-a923-19c5b3f6a46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccabf78999e642dd920d08158a295d38'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.428172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1375/2765] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:38:46 2022] DELETE /identity/v3/users/ccabf78999e642dd920d08158a295d38 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:46.656162 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:46.659573 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:46.671806 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e66cf-03c9-4caf-9e71-b64777cdae01 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:46.729969 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:46.730656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:46.730947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:46.731224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] PATH_INFO: `/v3/projects/cee5a7d58a774465910841f22a7b7813` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:46.732348 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee5a7d58a774465910841f22a7b7813)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:46.734321 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:46.905917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee5a7d58a774465910841f22a7b7813', 'request_id': 'req-afd7ee9a-c452-4173-847b-875ed618e341'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.973754 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee5a7d58a774465910841f22a7b7813', 'request_id': 'req-afd7ee9a-c452-4173-847b-875ed618e341'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.979885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee5a7d58a774465910841f22a7b7813', 'request_id': 'req-afd7ee9a-c452-4173-847b-875ed618e341'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.990715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] The token cache is being invalidate because project cee5a7d58a774465910841f22a7b7813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:38:46.990997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.991610 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-afd7ee9a-c452-4173-847b-875ed618e341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:38:46.993666 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1391/2766] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:38:46 2022] DELETE /identity/v3/projects/cee5a7d58a774465910841f22a7b7813 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:46.999402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.000671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.000671 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.019457 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:47.020684 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:47.077869 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1376/2767] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:47.085691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.086537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.086818 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.107778 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:47.108988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:47.116920 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1392/2768] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:47.120741 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.130207 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a083db-a740-4aee-b978-cbca804738c5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.134824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.135414 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.135694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.135968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.136617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:38:47.136920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.138569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.150635 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1377/2769] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:38:47.154697 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.163840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f249c2a-16f1-4e59-9e2c-6258e9ea2b77 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.169059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.169645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.169917 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.170187 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.170849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.172729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.272324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1393/2770] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:38:47 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:38:47.277010 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.287065 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5077313c-7cef-44c7-8580-15096e57c50c admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.342172 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.342825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.343099 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.343365 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.343993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.345917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.459737 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1378/2771] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:38:47 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:38:47.464559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.475032 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f219-e44c-4490-bcaa-fca4ff536b2e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.479606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.480264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.480541 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.480811 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.481453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.483081 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.493550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1394/2772] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:47.497484 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.508550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b25bec-6546-434d-a1d3-49b309c9be69 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.513325 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.513915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.514188 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.514491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.515214 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.516777 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21b7fd-c6b5-480e-a55d-5c8a1e54ae48 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.527942 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1379/2773] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:38:47.531916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.542068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab56b6-c385-4488-8124-9673cef1a7d8 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.546771 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.547435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.547721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.547987 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] PATH_INFO: `/v3/projects/ba7d87a416514ba191d510fbef59a669/users/94a9db9679914e7793ed1a48c224ec42/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.562761 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7d87a416514ba191d510fbef59a669, user_id=94a9db9679914e7793ed1a48c224ec42, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.565385 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.639577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1395/2774] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:38:47 2022] PUT /identity/v3/projects/ba7d87a416514ba191d510fbef59a669/users/94a9db9679914e7793ed1a48c224ec42/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:38:47.651970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.652260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.652536 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.660897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:47.680406 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] MFA Rules not processed for user `94a9db9679914e7793ed1a48c224ec42`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:38:47.681637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:47.746482 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1380/2775] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:38:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:38:47.758225 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:38:47.759947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:38:47.769891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:47.816648 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec32db5-5834-4265-818a-559a94e88c2b admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:38:47.820840 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:38:47.821443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:38:47.821717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:38:47.821983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:38:47.830151 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:38:47.831639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:38:47.841241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1396/2776] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:38:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:38:57.779378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=30 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:38:57.864982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=23 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:22.852760 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:22.862344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba83182-a5f1-4c7f-859c-fb3e20015d40 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:22.865895 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:22.866516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:22.866794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:22.867083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:22.868772 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:22.938605 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:22.939964 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84eeb9-0e1c-4d13-b6d9-2efd73c8f52e service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:22.951225 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1381/2777] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:39:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:39:29.070627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:29.136687 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7219b-c7ac-416e-ae40-05509b5b49a9 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:29.141192 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:29.141819 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:29.142098 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:29.142370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] PATH_INFO: `/v3/users/94a9db9679914e7793ed1a48c224ec42` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:29.143522 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a9db9679914e7793ed1a48c224ec42)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:29.145486 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:29.336647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a9db9679914e7793ed1a48c224ec42'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:29.350550 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a9db9679914e7793ed1a48c224ec42'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:29.360571 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a9db9679914e7793ed1a48c224ec42'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:29.361316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a9db9679914e7793ed1a48c224ec42'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:29.362328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a9db9679914e7793ed1a48c224ec42'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:29.438184 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1397/2778] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:39:29 2022] DELETE /identity/v3/users/94a9db9679914e7793ed1a48c224ec42 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:39:29.442723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:29.443272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:29.443272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:29.463348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:39:29.465148 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:29.525805 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1382/2779] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:39:29.536276 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:29.537722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:29.596293 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ff3f-9327-473f-b6c7-171f8ed4a3af admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:29.599893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:29.600493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:29.600773 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:29.601047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:29.610022 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:29.611511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:29.620874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1398/2780] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:39:30.035175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:30.036950 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:30.050224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaece4-d375-424d-990d-98bc41543cfd None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:30.059789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:30.059789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:30.059789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:30.059789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] PATH_INFO: `/v3/projects/ba7d87a416514ba191d510fbef59a669` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:30.060569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7d87a416514ba191d510fbef59a669)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:30.063412 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:30.194774 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7d87a416514ba191d510fbef59a669', 'request_id': 'req-c1911a1d-18e2-4819-9985-8d5a31f2877a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:30.203089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7d87a416514ba191d510fbef59a669', 'request_id': 'req-c1911a1d-18e2-4819-9985-8d5a31f2877a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:30.264140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7d87a416514ba191d510fbef59a669', 'request_id': 'req-c1911a1d-18e2-4819-9985-8d5a31f2877a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:30.277086 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] The token cache is being invalidate because project ba7d87a416514ba191d510fbef59a669 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:39:30.277377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:30.278035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:30.279644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1383/2781] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:39:30 2022] DELETE /identity/v3/projects/ba7d87a416514ba191d510fbef59a669 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:39:32.996025 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:32.997441 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:33.022956 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:33.053495 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c846b-cfea-48a8-9097-7741a8406a39 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:33.101294 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:33.101909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:33.102594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:33.103483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:33.105418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:33.174447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:33.175850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:33.186988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1399/2782] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:39:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:39:40.273018 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:43.223283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=64 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:47.722645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:47.724308 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:47.742092 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1911a1d-18e2-4819-9985-8d5a31f2877a admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:47.748066 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:47.748706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:47.749004 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:47.749272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:47.751111 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:47.764873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:47.766546 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:47.778891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1384/2783] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:39:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:39:57.773929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:58.059447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:58.060921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:58.126711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c605404-ae72-421f-8b1a-73d62b9a27f1 service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:58.131311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:58.131942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:58.132223 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:58.132506 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] PATH_INFO: `/v3/users/9561f323e125411bb83ecce21bb085c0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:58.133627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9561f323e125411bb83ecce21bb085c0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:58.135601 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:58.332749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9561f323e125411bb83ecce21bb085c0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.344663 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9561f323e125411bb83ecce21bb085c0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.351353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9561f323e125411bb83ecce21bb085c0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.352023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9561f323e125411bb83ecce21bb085c0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.352622 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-0fd991f0-abc1-4106-8b93-4065b654a67b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9561f323e125411bb83ecce21bb085c0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.419015 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1400/2784] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:39:58 2022] DELETE /identity/v3/users/9561f323e125411bb83ecce21bb085c0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:39:58.657677 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:58.681394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bac342-aa79-4772-a133-36ccb282978c service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:58.727492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:58.728116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:58.728390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:58.728659 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] PATH_INFO: `/v3/projects/fdee9665c45d41be9d34ba14def64757` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:58.730271 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdee9665c45d41be9d34ba14def64757)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:58.732244 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:58.840336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdee9665c45d41be9d34ba14def64757', 'request_id': 'req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.846103 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdee9665c45d41be9d34ba14def64757', 'request_id': 'req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.933604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdee9665c45d41be9d34ba14def64757', 'request_id': 'req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.945400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] The token cache is being invalidate because project fdee9665c45d41be9d34ba14def64757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:39:58.945685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.946299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-f6d0a77f-0518-4a89-8a7d-09fb28fcfa0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:39:58.947833 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1385/2785] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:39:58 2022] DELETE /identity/v3/projects/fdee9665c45d41be9d34ba14def64757 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:39:58.953668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:58.957975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:58.957975 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:58.975324 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:39:58.976525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:59.033214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1401/2786] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:39:59.041163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.041450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.041727 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.062008 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:39:59.063252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:59.071249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1386/2787] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:39:59.075418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.085115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b72e6-7efe-4eda-857f-0b97ab4d2e9c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.090211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.090841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.091117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.091391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.092038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:39:59.092340 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.093928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.105814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1402/2788] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:39:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:39:59.109858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.118930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48f15-30ad-4d85-9a0e-544122116c48 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.123599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.124211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.124488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.124757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.125386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.127317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.260253 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1387/2789] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:39:59.264914 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.275090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb48f0-f9fa-4f98-9412-4a631ba32c1c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.279205 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:59.330905 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.331502 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.331822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.332087 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.332782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.334967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.432706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1403/2790] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:39:59.436893 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.447326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec99f6-02e8-4d78-9063-b563a5efd673 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.452703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.453293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.453569 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.453840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.454509 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.456390 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.466820 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1388/2791] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:39:59.470583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.480328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0240415d-e1b1-4fef-8af1-43281a5acb55 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.484876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.485450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.485723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.485989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.486659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.488242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dac30-2630-45d9-988b-c596b6cd7e3c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.498699 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1404/2792] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:39:59.502378 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.512584 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfa146-66ec-4870-8b1a-d045e0830b3e admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.517079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.517707 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.517985 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.518254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] PATH_INFO: `/v3/projects/2cb369bfe0124a9996a52d35abafa8a1/users/84ffad5ed3de437eba2d10f4a708ef66/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.533599 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb369bfe0124a9996a52d35abafa8a1, user_id=84ffad5ed3de437eba2d10f4a708ef66, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.536278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.662622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1389/2793] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:39:59 2022] PUT /identity/v3/projects/2cb369bfe0124a9996a52d35abafa8a1/users/84ffad5ed3de437eba2d10f4a708ef66/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:39:59.673303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.673592 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.673874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.702587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] MFA Rules not processed for user `84ffad5ed3de437eba2d10f4a708ef66`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:39:59.703781 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:39:59.765942 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1405/2794] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:39:59.770374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.789683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:39:59.832428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275988c-26f7-4221-8dc5-e42601cced22 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.837295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.840595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.840595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.840595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.840595 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.841429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:39:59.916837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1390/2795] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:39:59.921406 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:39:59.932196 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c248b65-718a-435e-9a91-298f3dc76d7f None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:39:59.986465 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:39:59.987062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:39:59.987337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:39:59.987606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:39:59.988217 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:39:59.990115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.078789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1406/2796] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:39:59 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:40:00.082742 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.093031 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59814214-6f37-433f-b3d6-244140d82cbd admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.097670 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.098257 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.098571 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.098843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.099481 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.101074 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.111910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1391/2797] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:00.115848 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.122898 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:00.126816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2f08d-d8b5-4cf6-a7c2-f823ebbb0650 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.131525 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.132168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.137074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.137074 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] PATH_INFO: `/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e/users/24968f7952664cb088d471b53304df3d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.147288 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8ee8a13dcc47a6911b4a34d9154b0e, user_id=24968f7952664cb088d471b53304df3d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.149985 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.212716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1407/2798] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:00 2022] PUT /identity/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e/users/24968f7952664cb088d471b53304df3d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:00.217269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.227311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ada5e9-7a1c-4d44-9696-e40b6a2fb4ae admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.284045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.284706 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.284987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.285263 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.285902 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.287567 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f8ade-d22f-4755-93c9-687161f1345d admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.297617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1392/2799] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:00.301880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.312100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac18f96-b999-4092-b736-db8170c7f806 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.316902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.317553 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.317829 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.318100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] PATH_INFO: `/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e/users/24968f7952664cb088d471b53304df3d/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.320062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8ee8a13dcc47a6911b4a34d9154b0e, user_id=24968f7952664cb088d471b53304df3d, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.322762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.333460 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1408/2800] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:00 2022] PUT /identity/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e/users/24968f7952664cb088d471b53304df3d/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:00.345836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.346147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.346442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.373929 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] MFA Rules not processed for user `24968f7952664cb088d471b53304df3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:00.375186 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:00.441488 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1393/2801] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:00.453345 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.455062 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:00.512334 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9eb5-2f71-4e41-a0a6-784b9b3d077c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.516530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.516664 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.516976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.517152 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.526236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.527691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4b4fe-fc5b-4e33-a6fc-b7d17790f50d service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.537285 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1409/2802] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:00.566344 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:00.580740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:00.628650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26e5db-0912-4c02-9da9-5350ec85ab4c None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:00.633342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:00.633976 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.634252 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.634573 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] PATH_INFO: `/v3/users/84ffad5ed3de437eba2d10f4a708ef66` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.638073 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ffad5ed3de437eba2d10f4a708ef66)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:00.638310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:00.864191 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ffad5ed3de437eba2d10f4a708ef66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:00.871704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ffad5ed3de437eba2d10f4a708ef66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:00.947542 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ffad5ed3de437eba2d10f4a708ef66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:00.948118 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ffad5ed3de437eba2d10f4a708ef66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:00.960046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ffad5ed3de437eba2d10f4a708ef66'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:00.965935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1394/2803] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:00 2022] DELETE /identity/v3/users/84ffad5ed3de437eba2d10f4a708ef66 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:00.970602 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:00.970890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:00.971165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:00.990318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:00.991671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:01.049708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1410/2804] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:01.060167 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:01.061574 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:01.117535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad00781-2b4a-4163-bc7e-0c66a103efb0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:01.121763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:01.122353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:01.122663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:01.122934 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:01.131729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:01.133158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:01.142478 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1395/2805] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:01.568982 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:01.570678 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:01.583564 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ae95c-bced-4345-9d2a-0395e6caa9f5 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:01.588415 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:01.589715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:01.589990 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:01.590260 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] PATH_INFO: `/v3/projects/2cb369bfe0124a9996a52d35abafa8a1` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:01.591414 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb369bfe0124a9996a52d35abafa8a1)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:01.593318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:01.697000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb369bfe0124a9996a52d35abafa8a1', 'request_id': 'req-6506038e-abc0-4f55-a05a-1b9315eff502'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:01.755691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb369bfe0124a9996a52d35abafa8a1', 'request_id': 'req-6506038e-abc0-4f55-a05a-1b9315eff502'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:01.762143 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb369bfe0124a9996a52d35abafa8a1', 'request_id': 'req-6506038e-abc0-4f55-a05a-1b9315eff502'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:01.763286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:01.773586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] The token cache is being invalidate because project 2cb369bfe0124a9996a52d35abafa8a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:01.773870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:01.774645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:01.776694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1411/2806] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:01 2022] DELETE /identity/v3/projects/2cb369bfe0124a9996a52d35abafa8a1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:01.780629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:01.790479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9677e-6d23-43c2-bc3d-28bdb9c967d4 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:01.822507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:01.847145 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:01.847878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:01.848159 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:01.848454 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] PATH_INFO: `/v3/users/24968f7952664cb088d471b53304df3d` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:01.849664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24968f7952664cb088d471b53304df3d)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:01.851751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:02.061054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24968f7952664cb088d471b53304df3d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.067826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24968f7952664cb088d471b53304df3d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.136398 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24968f7952664cb088d471b53304df3d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.136968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24968f7952664cb088d471b53304df3d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.149040 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-041128c8-1a59-4776-a2d3-5186960cb0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24968f7952664cb088d471b53304df3d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.154900 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1396/2807] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:01 2022] DELETE /identity/v3/users/24968f7952664cb088d471b53304df3d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:02.563810 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:02.565247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:02.578224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506038e-abc0-4f55-a05a-1b9315eff502 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:02.636943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:02.637582 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:02.637863 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:02.638132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] PATH_INFO: `/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:02.639275 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c8ee8a13dcc47a6911b4a34d9154b0e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:02.641168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:02.761899 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c8ee8a13dcc47a6911b4a34d9154b0e', 'request_id': 'req-a0bbd988-9875-466f-8b78-fa4542f2177a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.818528 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c8ee8a13dcc47a6911b4a34d9154b0e', 'request_id': 'req-a0bbd988-9875-466f-8b78-fa4542f2177a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.824328 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c8ee8a13dcc47a6911b4a34d9154b0e', 'request_id': 'req-a0bbd988-9875-466f-8b78-fa4542f2177a'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.835438 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] The token cache is being invalidate because project 6c8ee8a13dcc47a6911b4a34d9154b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:02.836067 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.836380 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a0bbd988-9875-466f-8b78-fa4542f2177a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:02.837851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1412/2808] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:02 2022] DELETE /identity/v3/projects/6c8ee8a13dcc47a6911b4a34d9154b0e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:02.843205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:02.843491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:02.843768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:02.864199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:02.865404 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:02.923021 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1397/2809] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:02.932594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:02.932877 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:02.933146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:02.955112 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:02.956320 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:02.964719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1413/2810] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:02.968853 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:02.978543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ce888-e896-45e3-b413-d6dcba469fb9 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:02.983668 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:02.984264 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:02.984543 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:02.984814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:02.985461 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81568) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:02.985766 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:02.987409 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:02.998987 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1398/2811] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:40:03.002799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.012478 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b24d4-22ed-419a-b1c7-c7ebebed98b9 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.016997 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.017578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.017852 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.018496 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.019130 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.020999 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.102822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1414/2812] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:03 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:40:03.106980 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.117223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc5c02-1869-4ed8-b4e5-48b0b93356d0 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.157447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:03.173715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.174317 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.174632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.174904 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.175537 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.177468 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.284751 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1399/2813] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:03 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:03.288919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.304757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f8702-e1a0-44ea-93ae-6afbfb05f69f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.304757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.305024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.305136 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.305378 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.306017 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.307647 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.317396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1415/2814] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:03.321206 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.331694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9613d-a12e-46c4-9bca-2b3885881ae4 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.336448 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.337056 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.337337 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.337611 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.338254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.340466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a8260-c0e9-403b-b81b-c573211faf67 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.350832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1400/2815] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:03.354911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.364677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f270a1-0698-471e-9f5e-cbae2344bf49 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.369342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.370526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.370799 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.371066 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] PATH_INFO: `/v3/projects/569cdbbeec484e059c969432ebbf742d/users/06a05970d6c6453a830b5ac4c2ead463/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.385006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=569cdbbeec484e059c969432ebbf742d, user_id=06a05970d6c6453a830b5ac4c2ead463, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.388290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.455939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1416/2816] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:03 2022] PUT /identity/v3/projects/569cdbbeec484e059c969432ebbf742d/users/06a05970d6c6453a830b5ac4c2ead463/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:03.473696 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.473982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.474255 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.503591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] MFA Rules not processed for user `06a05970d6c6453a830b5ac4c2ead463`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:03.504841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:03.567183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1401/2817] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:03.571638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.600265 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:03.642167 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258391-1a72-42f9-9992-19540e0321c5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.647114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.647717 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.647996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.648298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] PATH_INFO: `/v3/projects` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.648949 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.650884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.768885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1417/2818] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:03 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:03.773027 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.783644 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c5bf-867c-4925-ab61-ad6f507e47ef None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.838455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.838970 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.839246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.839514 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.840150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.842044 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.938795 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1402/2819] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:03 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:03.947252 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.957548 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee35715-4c0d-4101-9ef2-0ef3e626814d admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.962140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.962837 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.963119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.963391 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:03.964060 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:03.965645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:03.975964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1418/2820] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:03.980329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:03.990312 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0419d4-8103-4376-ac25-f7eef5edc3b7 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:03.995504 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:03.996142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:03.996417 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:03.996682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] PATH_INFO: `/v3/projects/eac0c5843106473790350215247a1828/users/e603193f7a444474813d436b6f8db288/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:04.011305 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac0c5843106473790350215247a1828, user_id=e603193f7a444474813d436b6f8db288, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:04.013958 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:04.098645 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1403/2821] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:03 2022] PUT /identity/v3/projects/eac0c5843106473790350215247a1828/users/e603193f7a444474813d436b6f8db288/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:04.102543 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:04.113019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c6c3-aa62-4407-bcda-706019157816 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:04.168027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:04.168618 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:04.168894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:04.169160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:04.169809 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:04.171671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d7eb0-3ee3-4bcd-9be5-5d1dbcef24e3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:04.181234 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1419/2822] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:04.185440 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:04.195801 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828e5cc-3d6b-470f-9c41-6f3d305489ed admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:04.200342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:04.200986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:04.201262 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:04.201529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] PATH_INFO: `/v3/projects/eac0c5843106473790350215247a1828/users/e603193f7a444474813d436b6f8db288/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:04.203592 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac0c5843106473790350215247a1828, user_id=e603193f7a444474813d436b6f8db288, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:04.206299 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:04.216830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1404/2823] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:04 2022] PUT /identity/v3/projects/eac0c5843106473790350215247a1828/users/e603193f7a444474813d436b6f8db288/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:04.227916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:04.228200 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:04.228470 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:04.256176 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] MFA Rules not processed for user `e603193f7a444474813d436b6f8db288`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:04.257435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:04.323587 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1420/2824] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:04.334150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:04.335858 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:04.341682 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:04.392490 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8a25e-6469-41d7-8abf-76e910abd0eb admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:04.396154 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:04.396756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:04.397029 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:04.397301 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:04.407085 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:04.408524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9a8f8-b165-46dc-8a06-f2216772a08f service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:04.418428 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1405/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:04.933304 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:04.950000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:04.995055 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf593e3-b407-4969-b062-f0fd4327088e None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:04.999549 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:05.000169 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:05.000443 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:05.000712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] PATH_INFO: `/v3/users/06a05970d6c6453a830b5ac4c2ead463` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:05.002481 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a05970d6c6453a830b5ac4c2ead463)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:05.004392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:05.175316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a05970d6c6453a830b5ac4c2ead463'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.186983 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a05970d6c6453a830b5ac4c2ead463'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.193772 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a05970d6c6453a830b5ac4c2ead463'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.194476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a05970d6c6453a830b5ac4c2ead463'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.195082 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a05970d6c6453a830b5ac4c2ead463'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.275282 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1421/2826] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:04 2022] DELETE /identity/v3/users/06a05970d6c6453a830b5ac4c2ead463 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:05.280373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:05.280944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:05.280944 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:05.299585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:05.300783 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:05.358139 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1406/2827] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:05.367585 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:05.368981 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:05.425161 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3f85-85b0-412d-bad7-6a5648094146 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:05.428897 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:05.429433 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:05.429711 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:05.429984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:05.438816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:05.440236 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:05.449928 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1422/2828] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:05.860530 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:05.861990 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:05.874221 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46b99c-d656-41df-b2e8-ae8825f315d8 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:05.879143 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:05.879788 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:05.880064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:05.880328 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] PATH_INFO: `/v3/projects/569cdbbeec484e059c969432ebbf742d` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:05.881464 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=569cdbbeec484e059c969432ebbf742d)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:05.883540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:05.989971 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569cdbbeec484e059c969432ebbf742d', 'request_id': 'req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:05.995453 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569cdbbeec484e059c969432ebbf742d', 'request_id': 'req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.058147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569cdbbeec484e059c969432ebbf742d', 'request_id': 'req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.069171 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] The token cache is being invalidate because project 569cdbbeec484e059c969432ebbf742d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:06.069483 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.070135 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.071617 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1407/2829] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:05 2022] DELETE /identity/v3/projects/569cdbbeec484e059c969432ebbf742d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:06.076114 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:06.086059 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a676f-0ba4-496e-a156-eace78bf3fcb service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:06.118474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:06.140686 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:06.141303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:06.141576 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:06.141843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] PATH_INFO: `/v3/users/e603193f7a444474813d436b6f8db288` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:06.142970 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e603193f7a444474813d436b6f8db288)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:06.144870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:06.348825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e603193f7a444474813d436b6f8db288'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.360160 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e603193f7a444474813d436b6f8db288'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.367789 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e603193f7a444474813d436b6f8db288'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.368450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e603193f7a444474813d436b6f8db288'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.369137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-f624da4c-3d10-4b68-b427-389a99aadb80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e603193f7a444474813d436b6f8db288'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:06.435452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1423/2830] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:06 2022] DELETE /identity/v3/users/e603193f7a444474813d436b6f8db288 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:06.865525 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:06.867096 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:06.882215 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c3cdc-2c9c-4608-ae7f-b97b338e7b79 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:06.911079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:06.943034 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:06.944455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:06.944455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:06.944455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] PATH_INFO: `/v3/projects/eac0c5843106473790350215247a1828` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:06.945379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac0c5843106473790350215247a1828)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:06.947336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.051124 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac0c5843106473790350215247a1828', 'request_id': 'req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:07.057089 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac0c5843106473790350215247a1828', 'request_id': 'req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:07.129941 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac0c5843106473790350215247a1828', 'request_id': 'req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:07.141169 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] The token cache is being invalidate because project eac0c5843106473790350215247a1828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:07.141460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:07.142087 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-56ba0f1f-d995-47c0-a7ef-255b5a8bea87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:07.143626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1408/2831] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:06 2022] DELETE /identity/v3/projects/eac0c5843106473790350215247a1828 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:07.148660 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.149250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.149250 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.169530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:07.171041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:07.227072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1424/2832] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:07.235411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.235685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.235965 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.255424 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:07.256655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:07.264632 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1409/2833] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:07.268824 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.278921 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a82e49-c81d-42b1-bcf2-b0537d0a48ad None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.283579 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.284166 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.284444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.284727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.285393 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:07.285695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.287325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.299991 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1425/2834] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:40:07.303840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.313211 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09233b-8cde-4b87-bebc-617237d5db2f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.317948 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.318591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.318875 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.319151 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.319856 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.321763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.424394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1410/2835] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:07.428454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.438315 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86342b4-599c-45f6-be5d-036884181ca1 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.479594 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:07.493732 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.494333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.502155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.502155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.502155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.502155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.572831 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1426/2836] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:07.577564 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.587996 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb03ddc-fe3e-4e4c-bde0-f5d5109a7939 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.593341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.593930 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.594296 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.594594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.595237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.596840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.606899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1411/2837] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:07.610871 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.620893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e31d2d-3606-4cb8-b57b-bfec6fc03ac5 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.625627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.626211 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.626513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.626782 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.627420 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.629050 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce2ca-925d-4b55-89ee-8e273c5b9f30 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.641257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1427/2838] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:07.644629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.655134 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa03a7a-d123-4aa5-9f22-0d4d6b15b346 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.659691 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.660330 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.660604 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.660876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] PATH_INFO: `/v3/projects/2ae36572ef384b5c9f838c7e5a448464/users/4e4d2c73fec240c2b0a034775dce76ba/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.675274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae36572ef384b5c9f838c7e5a448464, user_id=4e4d2c73fec240c2b0a034775dce76ba, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.678025 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:07.749400 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1412/2839] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:07 2022] PUT /identity/v3/projects/2ae36572ef384b5c9f838c7e5a448464/users/4e4d2c73fec240c2b0a034775dce76ba/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:07.812856 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.813141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.813419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.841614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] MFA Rules not processed for user `4e4d2c73fec240c2b0a034775dce76ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:07.842847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:07.905208 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1428/2840] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:07.909568 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:07.970442 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:07.975397 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abbfe8-c3bc-454a-b86f-70b90abad5ee admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:07.980122 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:07.980717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:07.981028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:07.981294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:07.981912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:07.983839 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.062616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1413/2841] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:07 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:08.073557 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.085113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce211-5697-4cbd-84ae-6e995385a538 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.141085 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.141682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.141960 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.142231 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.142889 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.144836 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.251047 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1429/2842] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:08 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:08.254911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.265208 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60454f7-410f-4062-9678-db6bcbd842c2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.269823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.270437 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.270729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.271005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.271701 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.273285 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.283351 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1414/2843] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:08.287329 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.298185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fd4ad-3aec-46f5-b86e-4d961c836cd0 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.303006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.303683 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.303951 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.304224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] PATH_INFO: `/v3/projects/80ff3ed6f51f4b5599d78f7b26999861/users/813508e6a6544f198cb43804e5aee321/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.319727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ff3ed6f51f4b5599d78f7b26999861, user_id=813508e6a6544f198cb43804e5aee321, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.322430 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.402052 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1430/2844] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:08 2022] PUT /identity/v3/projects/80ff3ed6f51f4b5599d78f7b26999861/users/813508e6a6544f198cb43804e5aee321/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:08.405767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.415963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294db3d4-40e8-41b7-a5ae-4a65947a28b6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.471292 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.471936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.472218 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.472491 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.473127 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.474763 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5df74-8fc2-4d02-803a-4c98880bb5ad admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.485050 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1415/2845] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:40:08.489325 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.500248 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc64177-ad10-4426-8b38-3b00977fcc2c admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.504851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.505447 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.505724 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.505998 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] PATH_INFO: `/v3/projects/80ff3ed6f51f4b5599d78f7b26999861/users/813508e6a6544f198cb43804e5aee321/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.507979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ff3ed6f51f4b5599d78f7b26999861, user_id=813508e6a6544f198cb43804e5aee321, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.510636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.520657 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1431/2846] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:08 2022] PUT /identity/v3/projects/80ff3ed6f51f4b5599d78f7b26999861/users/813508e6a6544f198cb43804e5aee321/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:08.530363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.530714 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.530991 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.558860 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] MFA Rules not processed for user `813508e6a6544f198cb43804e5aee321`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:08.560643 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:08.627814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1416/2847] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:08.638642 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:08.640526 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:08.647128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:08.698521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaed404-c074-413a-ad2b-da8cc9ce8453 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:08.702439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:08.703046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:08.703322 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:08.703722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:08.758536 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:08.759893 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:08.768715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1432/2848] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:13.862205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:13.863652 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:13.878100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9b422-3413-4616-a112-274f5408f0a3 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:13.881651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:13.882251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:13.882562 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:13.882832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:13.897838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:13.897838 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5360-fc78-49ca-9bf1-24c40ee33d1b service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:13.903456 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1417/2849] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:18.764677 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=46 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:19.599636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:19.660841 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eac46e-2bf0-47a0-8b9e-adb9184cb97d service cinder] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:19.665511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:19.666168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:19.666482 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:19.666757 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] PATH_INFO: `/v3/users/4e4d2c73fec240c2b0a034775dce76ba` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:19.667922 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4d2c73fec240c2b0a034775dce76ba)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:19.670616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:19.842734 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4d2c73fec240c2b0a034775dce76ba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:19.854493 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4d2c73fec240c2b0a034775dce76ba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:19.861934 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4d2c73fec240c2b0a034775dce76ba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:19.862627 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4d2c73fec240c2b0a034775dce76ba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:19.863224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4d2c73fec240c2b0a034775dce76ba'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:19.954253 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1433/2850] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:19 2022] DELETE /identity/v3/users/4e4d2c73fec240c2b0a034775dce76ba => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:19.962216 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:19.962715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:19.962826 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:19.982939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:19.984136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:20.015377 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:20.041666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1418/2851] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:20.052229 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:20.053636 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:20.110327 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b44ad-c2a5-46c2-bb32-ca8ec32adcd4 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:20.114033 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:20.114673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:20.114950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:20.115219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:20.124140 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:20.125547 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:20.134808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1434/2852] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:20.504209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:20.504209 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:20.517418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30adc8e0-ee43-49bf-b631-8f6637582fc2 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:20.522812 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:20.524260 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:20.524550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:20.524830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] PATH_INFO: `/v3/projects/2ae36572ef384b5c9f838c7e5a448464` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:20.526370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae36572ef384b5c9f838c7e5a448464)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:20.528925 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:20.648095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae36572ef384b5c9f838c7e5a448464', 'request_id': 'req-7137b692-15ef-4ace-ad01-29f2603e8ba6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:20.653591 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae36572ef384b5c9f838c7e5a448464', 'request_id': 'req-7137b692-15ef-4ace-ad01-29f2603e8ba6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:20.726069 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae36572ef384b5c9f838c7e5a448464', 'request_id': 'req-7137b692-15ef-4ace-ad01-29f2603e8ba6'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:20.737197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] The token cache is being invalidate because project 2ae36572ef384b5c9f838c7e5a448464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:20.737552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:20.738177 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:20.739664 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1419/2853] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:20 2022] DELETE /identity/v3/projects/2ae36572ef384b5c9f838c7e5a448464 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:20.743715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:20.753723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193bc80-b911-4c40-87d6-f630a41de09c service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:20.771580 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:20.807662 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:20.808214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:20.808494 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:20.808764 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] PATH_INFO: `/v3/users/813508e6a6544f198cb43804e5aee321` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:20.810372 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813508e6a6544f198cb43804e5aee321)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:20.812382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:21.013513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813508e6a6544f198cb43804e5aee321'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.025383 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813508e6a6544f198cb43804e5aee321'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.032113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813508e6a6544f198cb43804e5aee321'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.032807 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813508e6a6544f198cb43804e5aee321'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.033409 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-099ddaf3-67e0-42e1-bacb-980eaa33a8c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813508e6a6544f198cb43804e5aee321'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.111996 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1435/2854] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:20 2022] DELETE /identity/v3/users/813508e6a6544f198cb43804e5aee321 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:21.546311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:21.547979 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:21.561059 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137b692-15ef-4ace-ad01-29f2603e8ba6 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:21.618923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:21.619559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:21.619843 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:21.620120 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] PATH_INFO: `/v3/projects/80ff3ed6f51f4b5599d78f7b26999861` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:21.621246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ff3ed6f51f4b5599d78f7b26999861)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:21.623379 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:21.737316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ff3ed6f51f4b5599d78f7b26999861', 'request_id': 'req-264f2db8-8461-4b9c-ac50-893d2b13ab73'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.742842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ff3ed6f51f4b5599d78f7b26999861', 'request_id': 'req-264f2db8-8461-4b9c-ac50-893d2b13ab73'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.746938 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:21.817718 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ff3ed6f51f4b5599d78f7b26999861', 'request_id': 'req-264f2db8-8461-4b9c-ac50-893d2b13ab73'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.829476 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] The token cache is being invalidate because project 80ff3ed6f51f4b5599d78f7b26999861 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:21.829957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.831083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-264f2db8-8461-4b9c-ac50-893d2b13ab73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:21.832572 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1420/2855] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:21 2022] DELETE /identity/v3/projects/80ff3ed6f51f4b5599d78f7b26999861 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:21.838749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:21.839100 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:21.839379 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:21.859257 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:21.860452 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:21.918449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1436/2856] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:21.927176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:21.927535 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:21.927821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:21.948402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:21.950133 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:21.958911 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1421/2857] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:21.963128 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:21.973209 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57a402-9481-43d0-a769-9661cfe5dfec None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:21.977750 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:21.978335 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:21.978671 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:21.978916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:21.979559 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:21.979857 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:21.981473 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:21.992716 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1437/2858] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:40:21.997028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.006386 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ba28b-f771-42f3-9bd2-8d527038487e None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.011110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.011699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.012505 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.012776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.013411 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.015360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.101403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1422/2859] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:21 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:22.105616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.115542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac8525-90f7-4e90-b95f-9abb32d272b7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.141858 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:22.171237 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.171835 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.172113 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.172384 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.173046 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.174992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.260423 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1438/2860] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:22 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:22.265197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.275540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b1c3b-832e-45d7-8400-c6d35257dac1 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.280289 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.280880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.281160 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.281431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.282070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.283703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.294466 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1423/2861] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:22.298278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.308422 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9171be-8440-4699-8866-dd4318af8355 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.313578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.314165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.314474 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.314751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.315395 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.316984 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66661ea-89e8-4bca-bedc-b8db3e75ba70 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.327605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1439/2862] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:22.331329 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.341275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd514c14-eec6-4781-94bf-5db10c3ba1ee admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.345851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.346521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.346799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.347068 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] PATH_INFO: `/v3/projects/38dfeab178c641a0a560f6ba42ed3135/users/4d0429f8f86844568db4ded118bc163b/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.361243 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38dfeab178c641a0a560f6ba42ed3135, user_id=4d0429f8f86844568db4ded118bc163b, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.363863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.428978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1424/2863] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:22 2022] PUT /identity/v3/projects/38dfeab178c641a0a560f6ba42ed3135/users/4d0429f8f86844568db4ded118bc163b/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:22.441790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.442076 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.442352 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.470561 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] MFA Rules not processed for user `4d0429f8f86844568db4ded118bc163b`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:22.471760 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:22.534292 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1440/2864] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:22.547147 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:22.548821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:22.604756 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ab123-d37c-419b-9c74-e8a8e22f315b admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:22.608918 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc91470011c746488ea56ca5d9e7a8e4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:22.609513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:22.609807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:22.610156 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:22.618612 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:22.619993 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c097be-4fed-4856-a0cd-d1066e8ae3c8 service glance] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:22.633915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1425/2865] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:32.531389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:32.625678 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:39.851839 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:39.911458 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f933974-e75e-40ea-86cb-bb721f9e371c None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:39.915825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:39.916434 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:39.916708 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:39.916976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] PATH_INFO: `/v3/users/4d0429f8f86844568db4ded118bc163b` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:39.918323 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0429f8f86844568db4ded118bc163b)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:39.920247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:40.118939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0429f8f86844568db4ded118bc163b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.241343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0429f8f86844568db4ded118bc163b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.241343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0429f8f86844568db4ded118bc163b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.241343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0429f8f86844568db4ded118bc163b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.241343 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0429f8f86844568db4ded118bc163b'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:40.272989 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1441/2866] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:39 2022] DELETE /identity/v3/users/4d0429f8f86844568db4ded118bc163b => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:40.277585 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:40.278142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:40.278282 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:40.297837 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:40.299098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:40.356269 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1426/2867] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:40.366142 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:40.367556 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:40.423511 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ff883-1136-4109-9a02-3cfb4ccc7b4f admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:40.427119 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:40.427705 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:40.427976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:40.428242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:40.437310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:40.438759 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050aa18-6134-4af0-91c6-66353a77c2e0 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:40.447952 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1442/2868] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:40.955279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:40.956710 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:40.971268 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41508cf-b924-47f5-9dd0-12b3be71451a None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:40.974876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:40.975513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:40.975824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:40.976201 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] PATH_INFO: `/v3/projects/38dfeab178c641a0a560f6ba42ed3135` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:40.977231 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38dfeab178c641a0a560f6ba42ed3135)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:40.979224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.131304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38dfeab178c641a0a560f6ba42ed3135', 'request_id': 'req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.136583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38dfeab178c641a0a560f6ba42ed3135', 'request_id': 'req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.221851 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38dfeab178c641a0a560f6ba42ed3135', 'request_id': 'req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.233224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] The token cache is being invalidate because project 38dfeab178c641a0a560f6ba42ed3135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:41.235326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.235326 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-3d7a826d-d17d-43d8-a2ec-d2063448b4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:41.235629 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1427/2869] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:40 2022] DELETE /identity/v3/projects/38dfeab178c641a0a560f6ba42ed3135 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:41.241203 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.241729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.241729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.262261 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.263578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.284165 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:41.335830 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1443/2870] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:41.343702 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.344224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.344224 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.365513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.366794 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.374829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1428/2871] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:40:41.379149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.388656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc724b65-120d-4335-a2c7-eebcb96206b4 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.393224 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.393808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.394083 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.394370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.395044 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:40:41.395368 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.396940 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.408307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1444/2872] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:40:41.413070 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.422825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c834-3996-4778-a5e6-ca5b540852a5 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.428449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.430840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.430840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.430840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.430840 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.432274 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.532910 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1429/2873] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:41.537694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.548510 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548de0-60e2-4d41-9507-c7ece288de62 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.604790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.605471 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.605749 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.606034 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.606719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.608777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.688992 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1445/2874] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:40:41.693121 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.703555 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44eb75-9533-4abe-b4a9-1a14d2ad8a69 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.708890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.709479 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.709757 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.710028 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.710726 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.712347 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.722376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1430/2875] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:41.726945 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.736925 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d5559-898b-448e-9223-7ac3591d53b7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.741635 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.742221 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.742537 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.742814 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.743450 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.745039 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf0c10-9c81-4cd0-8e58-bf610728ce4b admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.755444 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1446/2876] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:40:41.759463 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.769861 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfa9fd-f055-4d96-9e74-3537dec73f68 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:41.774382 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:41.775046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.775324 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.775622 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] PATH_INFO: `/v3/projects/d295b82c563e44338be0fa1b01f8a43f/users/4ea88e4a96254dd7b82596d9dba1507e/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.790277 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d295b82c563e44338be0fa1b01f8a43f, user_id=4ea88e4a96254dd7b82596d9dba1507e, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:41.793217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:41.856100 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1431/2877] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:40:41 2022] PUT /identity/v3/projects/d295b82c563e44338be0fa1b01f8a43f/users/4ea88e4a96254dd7b82596d9dba1507e/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:41.871606 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:41.871890 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:41.872213 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:41.899935 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] MFA Rules not processed for user `4ea88e4a96254dd7b82596d9dba1507e`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:41.901132 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.964751 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1447/2878] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:41.976506 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:41.978237 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:41.999972 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:42.034876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae62dd-d5df-404a-9273-3e6d21cd4b29 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.039754 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cb1f27ea5c49c9a805a9cdd678bef6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.040368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.040650 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.040933 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.049450 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.054560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63a154-eddf-41fb-bd7a-9fb990652b73 service cinder] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.059681 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1432/2879] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:42.085019 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.104072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:42.145419 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f343-ae40-4e68-8408-6e1cb7dcac25 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.149918 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.150566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.150842 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.151108 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] PATH_INFO: `/v3/users/4ea88e4a96254dd7b82596d9dba1507e` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.152222 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea88e4a96254dd7b82596d9dba1507e)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.154115 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.344715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea88e4a96254dd7b82596d9dba1507e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:42.357512 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea88e4a96254dd7b82596d9dba1507e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:42.367401 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea88e4a96254dd7b82596d9dba1507e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:42.368163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea88e4a96254dd7b82596d9dba1507e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:42.368816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea88e4a96254dd7b82596d9dba1507e'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:42.436979 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1448/2880] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:40:42 2022] DELETE /identity/v3/users/4ea88e4a96254dd7b82596d9dba1507e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:42.441984 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.442492 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.442606 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.462061 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:40:42.463402 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:42.524266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1433/2881] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:40:42.533792 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:42.535193 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:42.591365 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990859f-8bc1-4a9c-bc6a-fcb3ed1c9e57 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:42.595006 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:42.595540 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:42.595815 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:42.596090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:42.604825 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:42.606311 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:42.616038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1449/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:43.057968 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:43.066233 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:43.071346 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c8229-f744-42fe-ba94-d278d879fb04 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:43.075923 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:43.076552 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:43.076824 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:43.077098 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] PATH_INFO: `/v3/projects/d295b82c563e44338be0fa1b01f8a43f` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:43.078353 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d295b82c563e44338be0fa1b01f8a43f)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:43.080279 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:43.191163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd295b82c563e44338be0fa1b01f8a43f', 'request_id': 'req-d0f66c6c-9e20-438e-a58c-46190863150f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:43.196576 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd295b82c563e44338be0fa1b01f8a43f', 'request_id': 'req-d0f66c6c-9e20-438e-a58c-46190863150f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:43.276427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd295b82c563e44338be0fa1b01f8a43f', 'request_id': 'req-d0f66c6c-9e20-438e-a58c-46190863150f'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:43.288955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] The token cache is being invalidate because project d295b82c563e44338be0fa1b01f8a43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:40:43.289293 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:43.289953 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:40:43.291471 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1434/2883] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:40:43 2022] DELETE /identity/v3/projects/d295b82c563e44338be0fa1b01f8a43f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:40:51.478163 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:51.479721 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:51.491651 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d489760-0577-4a93-9c06-d924bf8990f2 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:51.529303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:51.539752 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:51.540431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:51.540625 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:51.540891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:51.542919 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:51.552874 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:51.554283 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:51.562370 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1450/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:53.285270 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:40:55.911594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:55.913136 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:55.925541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f66c6c-9e20-438e-a58c-46190863150f admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:55.929229 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:55.929832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:55.930104 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:55.930373 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:55.932116 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:55.942230 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:55.943699 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:55.951680 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1435/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:56.871851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:56.873567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.948849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640898be-8080-4f30-bdb8-d18b74ddf0a6 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:56.954673 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:56.955290 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:56.955570 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:56.955843 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:56.958071 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:56.970826 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:56.973000 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:56.983965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1451/2886] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:57.045854 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.055403 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a6f34-bc7a-4049-ac1f-e788fdfc91d0 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.059046 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.059651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.059924 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.060196 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.062825 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:57.082603 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.084035 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.097319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1436/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:40:57.183241 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:40:57.184667 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:57.252264 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9792a7-4dd3-457b-ac9f-45d33e62e0d7 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:40:57.257784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:40:57.258392 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:40:57.258695 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:40:57.258964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:40:57.260808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:40:57.273005 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:40:57.274615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:40:57.284462 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1452/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:40:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:41:07.090460 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=12 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:41:07.280138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=74 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:32.839447 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:32.841540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:32.860880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf697f0-b3e4-4016-ae36-d76394795d10 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:32.867822 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:32.868439 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:32.868863 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:32.869142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:32.870963 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:32.884249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:32.885926 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:32.897717 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1437/2889] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:39.048786 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:39.059707 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae5302-d70a-4e60-af69-a1593c6d0b9f service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:39.064813 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:39.065412 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:39.065691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:39.065967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:39.067727 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:39.079449 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:39.081018 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:39.091784 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1453/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:42.336873 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:42.347692 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9cc9c-756d-43e3-b532-1a0eb8e1bc26 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:42.352807 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:42.353704 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.354007 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.354276 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.356049 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:42.368294 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:42.369917 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:42.381067 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1438/2891] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:42.472024 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:42.472684 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:42.539487 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd08ce-5bfc-499b-9193-9f478c66916e service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:42.548560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:42.549185 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:42.549484 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:42.549753 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] PATH_INFO: `/v3/users/654c715e5d32487d863a65bbc77ff561` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:42.550902 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=654c715e5d32487d863a65bbc77ff561)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:42.552808 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:42.712256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654c715e5d32487d863a65bbc77ff561'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.724361 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654c715e5d32487d863a65bbc77ff561'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.731247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654c715e5d32487d863a65bbc77ff561'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.732432 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654c715e5d32487d863a65bbc77ff561'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.732588 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654c715e5d32487d863a65bbc77ff561'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:42.802207 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1454/2892] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:42 2022] DELETE /identity/v3/users/654c715e5d32487d863a65bbc77ff561 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:42:43.055897 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.067150 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c325882-3f47-4910-b0f0-13bf5cbb0969 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.125842 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.126507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.126880 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.127149 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] PATH_INFO: `/v3/projects/42a5c327025a4c04bde73c77acc4a452` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.128278 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a5c327025a4c04bde73c77acc4a452)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.130217 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.242144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a5c327025a4c04bde73c77acc4a452', 'request_id': 'req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.247508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a5c327025a4c04bde73c77acc4a452', 'request_id': 'req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.304962 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a5c327025a4c04bde73c77acc4a452', 'request_id': 'req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.316821 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] The token cache is being invalidate because project 42a5c327025a4c04bde73c77acc4a452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:43.317106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.317789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.319303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1439/2893] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:43 2022] DELETE /identity/v3/projects/42a5c327025a4c04bde73c77acc4a452 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:43.327656 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.329577 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:43.342527 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfd5aa-1670-4d57-baac-bec2e89c3b23 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.396894 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.397521 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.397798 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.398068 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] PATH_INFO: `/v3/users/2a11dfd4735b4c378340e122e01694b0` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.399214 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a11dfd4735b4c378340e122e01694b0)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.401117 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:43.406233 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:43.593137 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a11dfd4735b4c378340e122e01694b0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.604303 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a11dfd4735b4c378340e122e01694b0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.610939 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a11dfd4735b4c378340e122e01694b0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.611615 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a11dfd4735b4c378340e122e01694b0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.612269 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-a56f5d70-1c6d-4962-a03f-153da9b00e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a11dfd4735b4c378340e122e01694b0'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:43.693715 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1455/2894] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:42:43 2022] DELETE /identity/v3/users/2a11dfd4735b4c378340e122e01694b0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:43.912310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:43.913999 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:43.928832 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad4448-61fb-4ba4-bdcd-8ca1d967a028 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:43.991891 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:43.992649 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:43.993045 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:43.993427 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] PATH_INFO: `/v3/projects/f24d64ac4f064540a413be87910f83c2` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:43.995197 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f24d64ac4f064540a413be87910f83c2)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:43.997205 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.004559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:44.197429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f24d64ac4f064540a413be87910f83c2', 'request_id': 'req-c312bbef-a28f-46b0-b379-28fc1ca04d7c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:44.202666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f24d64ac4f064540a413be87910f83c2', 'request_id': 'req-c312bbef-a28f-46b0-b379-28fc1ca04d7c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:44.264396 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f24d64ac4f064540a413be87910f83c2', 'request_id': 'req-c312bbef-a28f-46b0-b379-28fc1ca04d7c'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:44.275399 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] The token cache is being invalidate because project f24d64ac4f064540a413be87910f83c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81568) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:42:44.279064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:44.279064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-c312bbef-a28f-46b0-b379-28fc1ca04d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:42:44.279064 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1440/2895] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:42:43 2022] DELETE /identity/v3/projects/f24d64ac4f064540a413be87910f83c2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:44.282795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.283572 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.283847 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.306230 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:44.307466 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:44.363916 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1456/2896] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:44.371485 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.371776 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.372079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.391370 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:44.392559 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:44.400310 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1441/2897] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:44.404186 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.414364 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dd60-5d23-4062-b49d-ce07dda9699d None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.418909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.419490 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.419766 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.420031 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.420668 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 05:42:44.420965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.422583 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.434394 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1457/2898] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 05:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 05:42:44.438616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.447939 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0a320-e5b5-43ee-81d8-b15703191fe6 None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.454521 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.455986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.456266 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.456541 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.457175 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.461637 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.541376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1442/2899] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:44.545670 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.556141 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa64452-497e-461e-95fe-c3145d7bf527 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.611696 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.612387 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.612560 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.612851 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.613463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.615453 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.706431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1458/2900] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 05:42:44.709804 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.720275 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540fc5c6-b4ed-423e-9d36-867d8a6bc440 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.725548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.726146 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.726455 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.731899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.731899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.731899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.739043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1443/2901] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:44.742822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.748092 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:44.753318 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b88d29-c215-41cc-87ae-f086630df4d7 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.757884 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.760645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.760645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.760645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.760645 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.761373 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512643c-ca14-41b5-b0c1-173479d678a3 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.773578 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1459/2902] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:44.777796 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.788915 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389dde29-8803-47aa-9e74-8fad832a9b83 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:44.794015 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:44.794683 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.794960 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.795228 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] PATH_INFO: `/v3/projects/5b65e133d42c4ffa9ee97e8a21bb4a7a/users/32df71a35841485ba9166d0cad36755d/roles/2b04b45f17f1428facf4bd2291e8bcb3` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.809368 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b65e133d42c4ffa9ee97e8a21bb4a7a, user_id=32df71a35841485ba9166d0cad36755d, role_id=2b04b45f17f1428facf4bd2291e8bcb3)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:44.812202 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:44.893855 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1444/2903] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:44 2022] PUT /identity/v3/projects/5b65e133d42c4ffa9ee97e8a21bb4a7a/users/32df71a35841485ba9166d0cad36755d/roles/2b04b45f17f1428facf4bd2291e8bcb3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:44.898965 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c596-da8d-469f-b804-23ca41f40ffa None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:44.899247 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c596-da8d-469f-b804-23ca41f40ffa None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:44.899542 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955c596-da8d-469f-b804-23ca41f40ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:44.919351 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-8955c596-da8d-469f-b804-23ca41f40ffa None None] MFA Rules not processed for user `ad7989abb40e4e979f3e309692f0a847`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:44.920598 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8955c596-da8d-469f-b804-23ca41f40ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:44.976959 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1460/2904] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:44.986321 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:44.988054 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:45.043144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1969-d5ee-44ad-8377-9d9f0bfa41b2 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:45.046782 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:45.047374 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:45.047651 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:45.047920 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:45.056829 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:45.058306 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:45.068272 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1445/2905] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:42:50.313586 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.314170 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.314279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.342950 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] MFA Rules not processed for user `32df71a35841485ba9166d0cad36755d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:50.344962 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:50.418880 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1461/2906] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:42:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:50.423912 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.433747 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac1aef-355b-4103-839b-30f1f2e13af2 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.438336 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.438986 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.439295 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.439540 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] PATH_INFO: `/v3/projects` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.440176 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.442158 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.512616 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1446/2907] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 05:42:50 2022] POST /identity/v3/projects => generated 379 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:50.516723 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.527431 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df229811-ab58-4b10-bb76-e2be465fe9de None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.584746 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.585360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.585638 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.585909 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] PATH_INFO: `/v3/users` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.586599 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.588513 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.653816 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1462/2908] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 05:42:50 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 05:42:50.658733 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.669314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af8310-b71d-4f23-b49f-1159afbee594 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.673936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.674560 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.674882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.675152 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.675789 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.677389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.683311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:50.688164 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1447/2909] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 05:42:50.692299 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.702567 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d56403-d8c1-470d-a767-6bb9e4231652 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.707691 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.708358 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.708639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.708913 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] PATH_INFO: `/v3/projects/25683106027e46afa7f362c2f66c016c/users/05976dd26b0348eba776fd5dbded9d01/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.717605 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:42:50.724643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25683106027e46afa7f362c2f66c016c, user_id=05976dd26b0348eba776fd5dbded9d01, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.727397 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.802719 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1463/2910] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:50 2022] PUT /identity/v3/projects/25683106027e46afa7f362c2f66c016c/users/05976dd26b0348eba776fd5dbded9d01/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:50.812786 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.818768 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac105e5-c274-4f8d-ad1e-afbfc6ccbf17 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.877666 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.878283 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.878593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.878882 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.879524 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.881241 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfec4-e5c7-450d-85b0-87c728cc1a1b admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.892656 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1448/2911] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 05:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 05:42:50.896971 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:50.909389 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de23150-5209-4585-a9e3-56e5306e315e admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:50.923220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:50.923220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:50.923220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:50.923220 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] PATH_INFO: `/v3/projects/25683106027e46afa7f362c2f66c016c/users/05976dd26b0348eba776fd5dbded9d01/roles/875fac1137784b2fb2938ba0c2c98083` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:50.924694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25683106027e46afa7f362c2f66c016c, user_id=05976dd26b0348eba776fd5dbded9d01, role_id=875fac1137784b2fb2938ba0c2c98083)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:50.924694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:50.933367 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1464/2912] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 05:42:50 2022] PUT /identity/v3/projects/25683106027e46afa7f362c2f66c016c/users/05976dd26b0348eba776fd5dbded9d01/roles/875fac1137784b2fb2938ba0c2c98083 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:42:55.992303 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b538645-5baa-436c-b578-82d3c02a08b7 None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:55.992781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b538645-5baa-436c-b578-82d3c02a08b7 None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:55.992781 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b538645-5baa-436c-b578-82d3c02a08b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:56.024418 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-4b538645-5baa-436c-b578-82d3c02a08b7 None None] MFA Rules not processed for user `05976dd26b0348eba776fd5dbded9d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:42:56.025715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4b538645-5baa-436c-b578-82d3c02a08b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:56.097827 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1449/2913] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:42:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:42:56.113876 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:42:56.115747 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:42:56.192617 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86ed53-b1d9-450e-8ce7-c0386bce1873 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:42:56.198694 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:42:56.199326 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:42:56.199616 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:42:56.199891 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:42:56.260530 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:42:56.262145 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:42:56.271706 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1465/2914] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:42:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:06.095246 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=63 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:06.267505 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:14.947850 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:14.949166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:14.949166 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:14.967898 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] MFA Rules not processed for user `32df71a35841485ba9166d0cad36755d`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:43:14.969083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:14.975936 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1450/2915] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:43:24.962548 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=3 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:32.491529 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:32.547072 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b290d1d-e7d6-4991-bba8-f903e6676d74 service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:32.550639 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:32.551242 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:32.551519 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:32.551790 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:32.561179 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:32.562650 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:32.572439 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1466/2916] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:34.389005 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:43:34.401977 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166c5e9-c0a8-422f-ab16-23fa505afaec None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:43:34.407694 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:43:34.408363 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:43:34.408703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:43:34.409055 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:43:34.410834 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:43:34.423520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:43:34.425183 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:43:34.436583 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1451/2917] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:43:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:43:42.567712 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=26 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:43:44.431982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:30.104278 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:30.105812 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:30.117037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdefffc-e32f-4991-94f3-c0f78474b15a service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:30.121476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:30.122073 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:30.122353 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:30.122741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:30.124446 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:30.137986 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:30.142156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:30.150652 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1467/2918] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:31.330458 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:31.339720 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b353a-be28-430e-a510-ec7794c329ac service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:31.343254 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:31.343836 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:31.344109 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:31.344376 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:31.346093 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:31.360669 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:31.362140 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57867b-1ed5-4c99-a940-4f985b6f2d33 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:31.373715 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1452/2919] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:32.737489 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:32.739043 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:32.754178 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcf7d67-a3e6-40d1-93eb-e2ed3dd0a12c service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:32.761333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:32.761964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:32.762246 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:32.762566 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:32.764337 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:32.778155 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:32.779822 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:32.807382 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1468/2920] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:34.801935 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] REQUEST_METHOD: `POST` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:34.802431 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:34.802539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:34.820955 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.auth.core [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] MFA Rules not processed for user `32df71a35841485ba9166d0cad36755d`. Rule list: `[]` (Enabled: `True`). {{(pid=81568) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:45:34.822142 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:34.828899 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1453/2921] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 05:45:35.141201 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:35.151454 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff9187-4e55-45f1-84eb-3a8148da0bbc service nova] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:35.155156 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:35.155741 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:35.156014 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:35.156279 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:35.157964 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:35.167256 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:35.168658 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495995cd-ab81-436c-ac1e-2d1610bdff22 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:35.176374 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1469/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:45:44.816079 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=9 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:45.172885 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=18 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:45:55.855729 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:45:55.865394 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef792013-e66e-473e-af90-e60e5b44ed8f None None] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:45:55.868954 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:45:55.869547 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:45:55.869823 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:45:55.870090 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:45:55.871841 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:45:55.881627 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:45:55.883097 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:45:55.891623 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1454/2923] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:45:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 05:46:05.887467 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=6 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:34.491346 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] REQUEST_METHOD: `POST` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:34.491870 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:34.491988 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:34.522689 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.auth.core [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] MFA Rules not processed for user `32df71a35841485ba9166d0cad36755d`. Rule list: `[]` (Enabled: `True`). {{(pid=81569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 05:47:34.523906 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:34.546795 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1470/2924] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 05:47:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 05:47:34.872539 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:34.891496 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c59de3-5d0c-472d-be3e-bef96d3fe577 service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:34.895311 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:34.895903 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:34.896235 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:34.896508 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:34.898239 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:34.917095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:34.918594 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:34.926449 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1455/2925] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:35.637742 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:35.638845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:35.740926 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26268fff-5eb1-4301-b4e0-ba688be1da82 None None] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:35.746023 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:35.746661 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:35.746943 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:35.747219 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:35.748958 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:35.760463 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:35.762041 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:35.772659 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1471/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:36.780465 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:36.783319 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:36.805297 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6809136-5c96-49f4-a59e-e04a742883aa service neutron] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:36.810550 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:36.811163 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:36.811554 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:36.811835 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:36.813586 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:36.815777 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:36.826890 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:36.827244 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b899949-4a15-41b3-865c-011653a4603f service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:36.828422 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:36.832289 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:36.832875 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:36.833149 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:36.833418 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:36.839043 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1456/2927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:36.842850 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:36.844435 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:36.855896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1472/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:47:46.834199 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=21 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:46.851090 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=65 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:47:51.208992 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:47:51.228137 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08de7c1-cffe-4de3-98f7-8c752e202e15 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:47:51.233498 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:47:51.234117 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:47:51.234391 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:47:51.234723 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:47:51.236420 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:47:51.250249 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:47:51.251876 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:47:51.263452 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1457/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:47:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:48:01.258830 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=9 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:23.992881 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:24.013342 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97878eb-a155-450c-bf20-2f4d3fa10080 service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:24.016896 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:24.017483 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:24.017758 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:24.018027 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:24.026168 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:24.027614 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:24.036255 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1473/2930] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:32.121516 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:32.147487 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f7c19-ca61-4685-aedf-b5ded0641965 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:32.153703 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a81fbda6880a41be8922a2f8978e7f21', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:32.154383 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:32.154758 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:32.155101 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:32.167429 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:32.169105 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:32.182654 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1458/2931] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:34.032976 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=10 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:38.177298 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:38.186206 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b40819-2c36-41dc-8d1b-38cebe2a65bf service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:38.189849 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d00e1ef6c1a4f63a9a4514aac586c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:38.190461 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:38.190738 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:38.191002 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:38.258703 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:38.260390 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:38.278569 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1474/2932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:41.897767 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:41.908982 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773c1cd-e2ca-4162-8664-9d42f3d3db08 service nova] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:41.917359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:41.917359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:41.917359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:41.917359 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:41.917878 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:41.928361 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:41.929957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:41.940320 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1459/2933] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:45.251845 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:45.264722 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e28220-05b8-40cf-8b77-f4b87cc8ce92 service neutron] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:45.270103 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:45.270762 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] REQUEST_METHOD: `GET` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:45.271038 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:45.272037 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:45.273801 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:45.286476 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:45.288911 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:45.302396 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1475/2934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:49:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:49:45.376316 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:45.388063 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bc576-2b26-4ba6-9cad-7f6a27635d38 service placement] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:45.392738 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:45.393389 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:45.393663 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:45.393957 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] PATH_INFO: `/v3/users/32df71a35841485ba9166d0cad36755d` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:45.395095 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32df71a35841485ba9166d0cad36755d)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:45.397565 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:45.565423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32df71a35841485ba9166d0cad36755d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:45.573341 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32df71a35841485ba9166d0cad36755d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:45.642062 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32df71a35841485ba9166d0cad36755d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:45.646371 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32df71a35841485ba9166d0cad36755d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:45.655724 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32df71a35841485ba9166d0cad36755d'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:45.661580 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1460/2935] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:49:45 2022] DELETE /identity/v3/users/32df71a35841485ba9166d0cad36755d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:45.901729 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:45.913094 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec23ab-e77a-4fbf-bcc9-99498401d3e7 service placement] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:45.970424 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:45.971056 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:45.971333 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:45.971597 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] PATH_INFO: `/v3/projects/5b65e133d42c4ffa9ee97e8a21bb4a7a` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:45.972698 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b65e133d42c4ffa9ee97e8a21bb4a7a)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:45.974640 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:46.071360 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b65e133d42c4ffa9ee97e8a21bb4a7a', 'request_id': 'req-563d2b4d-cf14-431b-a96d-2f405a8bc630'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.186675 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b65e133d42c4ffa9ee97e8a21bb4a7a', 'request_id': 'req-563d2b4d-cf14-431b-a96d-2f405a8bc630'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.192310 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b65e133d42c4ffa9ee97e8a21bb4a7a', 'request_id': 'req-563d2b4d-cf14-431b-a96d-2f405a8bc630'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.194026 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:46.215886 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] The token cache is being invalidate because project 5b65e133d42c4ffa9ee97e8a21bb4a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:49:46.216174 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.216791 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.218286 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1476/2936] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:49:45 2022] DELETE /identity/v3/projects/5b65e133d42c4ffa9ee97e8a21bb4a7a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:46.222304 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:46.223740 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:46.237507 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31610e5e-0a57-4b70-9e24-74e4be88ba53 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:46.293314 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:46.293937 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:46.294213 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:46.294513 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] PATH_INFO: `/v3/users/05976dd26b0348eba776fd5dbded9d01` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:46.295658 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05976dd26b0348eba776fd5dbded9d01)` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:46.297593 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:46.450342 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:46.456106 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05976dd26b0348eba776fd5dbded9d01'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.462978 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05976dd26b0348eba776fd5dbded9d01'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.550520 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05976dd26b0348eba776fd5dbded9d01'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.551144 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05976dd26b0348eba776fd5dbded9d01'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.564534 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.notifications [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05976dd26b0348eba776fd5dbded9d01'} {{(pid=81568) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:46.570348 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1461/2937] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 05:49:46 2022] DELETE /identity/v3/users/05976dd26b0348eba776fd5dbded9d01 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 05:49:46.826682 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Authenticating user token {{(pid=81569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:49:46.828146 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.fernet_utils [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:49:46.841354 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d2b4d-cf14-431b-a96d-2f405a8bc630 admin admin] Validating token access rules against request {{(pid=81569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:49:46.897239 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad7989abb40e4e979f3e309692f0a847', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd94b8179b9d4ba6806f99509678d7fb', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:49:46.898362 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:49:46.898674 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:49:46.898947 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] PATH_INFO: `/v3/projects/25683106027e46afa7f362c2f66c016c` {{(pid=81569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:49:46.900061 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25683106027e46afa7f362c2f66c016c)` {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:49:46.901967 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] RBAC: Authorization granted {{(pid=81569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:49:47.027138 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25683106027e46afa7f362c2f66c016c', 'request_id': 'req-40e69944-23c9-4424-8218-255cadf8fc07'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.089013 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25683106027e46afa7f362c2f66c016c', 'request_id': 'req-40e69944-23c9-4424-8218-255cadf8fc07'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.099096 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25683106027e46afa7f362c2f66c016c', 'request_id': 'req-40e69944-23c9-4424-8218-255cadf8fc07'} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.110316 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] The token cache is being invalidate because project 25683106027e46afa7f362c2f66c016c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 05:49:47.110643 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.111307 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG keystone.notifications [None req-40e69944-23c9-4424-8218-255cadf8fc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 05:49:47.112796 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: [pid: 81569|app: 0|req: 1477/2938] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 05:49:46 2022] DELETE /identity/v3/projects/25683106027e46afa7f362c2f66c016c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 05:49:56.632223 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:49:57.107270 dv-cinder-836027-8-iscsi devstack@keystone.service[81569]: DEBUG dbcounter [-] [81569] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 05:52:40.997750 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Authenticating user token {{(pid=81568) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 05:52:40.999423 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:52:41.068360 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21120859-4f88-4cc7-b53e-bf112864f638 admin admin] Validating token access rules against request {{(pid=81568) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 05:52:41.124655 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c57f8b0816a4832b24043daea93e113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27a422cf07fd4997b80019272e0addc0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81568) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 05:52:41.125251 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] REQUEST_METHOD: `GET` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 05:52:41.125529 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] SCRIPT_NAME: `/identity` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 05:52:41.125799 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81568) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 05:52:41.127626 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.fernet_utils [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81568) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 05:52:41.139110 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 05:52:41.140685 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe2b8d-8224-42ff-aa66-6ce37d57c482 service placement] RBAC: Authorization granted {{(pid=81568) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 05:52:41.151083 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: [pid: 81568|app: 0|req: 1462/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 05:52:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 05:52:51.146814 dv-cinder-836027-8-iscsi devstack@keystone.service[81568]: DEBUG dbcounter [-] [81568] Writing DB stats keystone:SELECT=58 {{(pid=81568) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}